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ABSTRACT 

Part of an investigation of the effectiveness of 
Education Consolidation and Improvement Act (ECIA) Chapter 1 reading 
programs, this document reports on a study of 699 second and fourth 
grade students in Chapter 1 programs in Iowa. An introduction notes 
that the study was conducted to discover instructional 
characteristics that affect student achievement, including kinds and 
amounts of instruction. It also explains the objectives and 
management plan of the study. The second and third sections detail 
the study rationale and design, while a fourth section outlines the 
instrumentation of the study, including types of instructional time, 
site selection, and teacher orientation. The fifth section describes 
results of the study, noting how students and teachers utilized time, 
and offering structural and contextual Chapter 1 program 
characteristics. The sixth section, divided into two parts for second 
and fourth grade students, offers analytical findings. Finally, a 
discussion section notes that simply increasing instructional time 
did not improve achievement, that student achievement increased with 
some time devoted to off-task activities, and that achievement 
increased, particularly for second graders, /hen teachers spent 
adequate cunounts of time on organizing and managing the classroom. 
(Nineteen tables, the Study Observer Handbook, and definitions of 
variables are included.) (JC) 
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INTROXJCriON 



Secretary's Initiative 

In 1983, The Secretary's Initiative to Improve the Quality of Chapter 1, 
ECIA Projects was implemented by the United States Department of Education. 
This effort authorized the allocation of grant funds to State Departments of 
Education to assist in identifying and impLementing successful Chapter 1 
program components. The Chapter 1 Section of the lowa Department of Public 
Instruction was awarded one of the Secretary' s Initiative grants. Tnis report 
contains the results of work performed fcy the Iowa Chapter 1 Section under tlie 
Initiative. 

Icwa Initiative Project 

The intent of the Icwa Secretary's Initiative Project was to conduct an 
on-site study of within-class instructional characteristics that affect student 
achievanent, including the kinds and amounts of instruction that occur in Iowa 
Chapter 1 classrooms. Through the results of this investigation, it was hoped 
to explain wliy some Chapter 1 projects demonstrate consistently higher or lower 
achievement gains. 

For several years, the Icwa Chapter 1 Section had been attempting to 
inves ^gate the factors that contribute to consistent high or low achievement 
performance among its districts' Chapter 1 reading programs. Efew relationsliips 
had been found, possibly due to the fact that global building-level measures of 
program and instructional characteristics were used and these building-level 
measures did not vary greatly at such an aggregate level of analysis. This 
lack of findings is in keeping with criticisms of "school effects" literature, 
which has frequently been criticized for its failure to acknowledge that 
qualitative variations in the educational experiences of students within a 
school setting can have important effects (Hauser et al., 1976; Alexander, Cook 
and McDill, 1978; Griffin and Alexander, 1978) . 

Project Objectives 

The fdlcwing were identified as the dDjectives of this project: 

1. To improve the quality of Chapter 1 reading programs; 

2. To identify within-class variables at the classroom level that account 
for individual student reading achievement gains, or that impede the 
effectiveness of Chapter 1 programs; 

3. To Identify within-class variables at the classroom level that enhance 
or impede individual student time-on-task; 

4. To identify instructional appro: -'es of teachers that characterize 
effective Chapter 1 reading programs; 
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5. To plan for the dissemination of findings through the provision of 
technical assistance to Local Education Agencies (LEAs) in order to 
(a) assess their own adapter 1 reading programs on the identified 
effectiveness dimensions, and (b) to develop a strategy for improving 
ineffective reading classroon management techniqueSf time management 
and instructional practices. 

Project Staff and Management Plan 

The Table of Organization for this project is shown in Figure 1 be].ow. 

FiaiRE 1 
IOWA READING SIUDY 
TABLE OF OPGANIZAl^ICN 



Director 
Dr. Oliver Hlmley 





Asdlstsnt Project Director 




Coleen HcClanahan | 



TAC Technical 
Advisor 
Judy Pfannenstlel 



TAC Technical 
Advisor 
Dr. Carol Vacek 



TAC Technical 
Advisor 
Dr. Diane Seltzer 



Principal Analyst 
Dr. LeJand I'rk 



Assistant Analyst 
Dr. David Alvord 



On-site 
Observers 



The organizational plan for the iranagement of this project included several 
individuals who have been involved in Iowa Chapter 1 evaluation and program 
improvement efforts over an extended period of time. Therefore, each was 
extremely knowledgeable of past e/aluation efforts as well and the direction in 
which the state wished to continue. A more complete discussion of each of the 
personnel involved is contained in Appendix A, Qualifications and 
Responsibilities of Key Personnel. 



gnJDY RATIONALE 

Past research findings have indicated that kinds and amounts of instruction 
may be vastly different for different students in the same classroom. Several 
reasons for anticipated variation within schools and within classes exist. One 
popular notion is that the degree of learning that takes place is a function of 
the quantity of learning activity. 

Instructional Time 



Research findings substantiating the relationship between a student's 
allocated instructional time and achievement have enhanced the importance of 
instructional time in models of achievement (Frederick and Walberg, 1980; 
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Denhsni and Liebermarir 1980; Filfcy and Cahanr 1978; Blockr 1975; Barr, 1975, 
1974; AndersoHf 1973; Harris and Serwerr 1966). Hcweverr the modest 
relationships demonstrated have led to subsequent conclusions that the 
theoretically desirable variable for measurement is engaged timer not merely 
a llocated or attended time. The relationship between task engagement (engaged 
ime) and achievement has received empirical support fran a nunber of recent 
observational studies (Sirotnikr 1982; Leinhardtr Zignondr and Codeyr 1981; 
Evertson, 1980; Powell and Dishawr 1980; Stallings, 1980 r 1975; Pfannenstiel 
and Sewellr 1980; Good and Beckermanr 1978; Arlin and Roth, ]978; Carroll and 
Spearitt, 1977). 

Teacher Interactions and Kinds of Instruction 

Teachers generally are unaware o£ the specific ways in which their 
interaction with students influences their students' classroom behavior with 
regard to task engagement and, ultimatelyr achievement. Certainly, the mture 
of the teaclier's own behavior may affect the variation that exists within a 
classroom regarding time-on-task and achievement outcomes. Seme teachers have 
been found to interact frequently with students, while others rarely interact 
(Jacksonr 1968) . Lack of much on-task student-teacher interaction has been 
found to be negatively related to gains (Stallingsr Needles and Stayrock, 
1979) , while a greater amount of student teacher interaction has been found 
positively related to gains, although the nature of this interaction is not 
always made dear. 

Not only the quantity of teacher interactions but also the quality of these 
interactions has demonstrated a relationship to adiievement. Research has 
shewn that a supportive environment where frequent positive feedback is 
provided for appropriate bdiavior is more conducive to student achievement than 
ic an enviroment where disapproval is the primary feedback. Furthermorer 
there has been sane evidence that teachers tend to respond more favorably, 
provide more praiser and are more st^^^rtive of hi^achieving than of 
low-achieving students. The recent finding that a hig^i percentage of teacher 
feedback was concentrated in the more negative categories led to the suggestion 
that "many teachers need to acquire better dassroan and behavior maiiaganent 
skills" (Thanpsonr White and Morgan, 1982 : 234). 

Studies of teacher use of classrocm time echo this reconmendation. 
Thanpson et al. (1982) found that about one-half of the teacher's time was 
spent on academic activities with the remaining tiine spent on management or 
behavioral activities. Sirotnik (1982) foimd similar levels of non-task 
activities, and recormended that future "quantity of schooling" studie^ include 
time spent on discipline and control as well as time-on task as areas of 
investigation. 

Kinds of instruction in the regular classroon have been the subject of much, 
inquiry, and have been defined in a nunber of ways and at varying levels of 
specificity. Areas such as instructional content, teacher behaviors, materials 
used, social dimate, ph/sical arrangement c£ classrooms, grouping, etc, have 
been included in the definitions. However, kinds of instruction have teen 
largely uninvestigated in Chapter 1 programs specially in terms of observed 
interactive behaviors between teachers and students. In addition, the 
quantitative and qualitative aspects of teacher instructional approaches have 
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not been investigated because observational studies have focused mostly on 
either student behaviors or teacher behavior s. Very infrequently have the 
interactive effects of teacher and student behaviors been observed. Both the 
apount of teacher interaction at the individual student level, and the 
instructional qualities of that interaction were areas of interest in the 
present study. 

STUDY DESIGN 

Ttie Iowa Chapter 1 reading study was designed to acconplish the following: 

1 . Obtain estimates of the nunber of hours of Chapter 1 e)?x)sure a 
typical Chapter 1 student receives between pretesting and posttesting; 

2. Generate descriptive information that characterizes how students and 
teaciiers spend their Chapter 1 class time, cind to prcvide answers to 
the following questions: 

A. For what proportion of class time is the typical Chapter 1 
student actively engaged in reading-related tasks vs. non-task 
behavior? 

B. What are students doing when they are not task-engaged? 

C. For what proportion of class time is the typical Chapter 1 
teacner interacting with students in reading instruction and/or 
reading related tasks vs. rK)n- instructional tasks? 

D. What are teachers doing when they are not engaged in reading 
instruction and/or reading-related tasks; 

i . To investigate the relationship between kinds of teacher instructional 
interaction and student time-on-task (e.g.^ Do the ways in which 
students and teachers spend their task ta a influence the amount of 
time that studtnts spend on reading-related tasks?); 

4. To investi^te the relationship between a student's reading 

acWcvement Cind (I) the amount of exposure to reading instruction 
between pretest aix3 posttest, (2) the proportion of class time in 
which students are task-engaged, and (3) the proportion of the total 
instructional time utilized for specific on- task behaviors; and 

^j. To examine at the group level the factors that distinguish effective 
reading classes frcm ineffective classes in terms of teacher 
Lurhavioru, student behaviors and general program characteristics. 

T^itjoretioal E)y?ctattons 

r tee aBCimptions about effective reading instruction guided the design of 
XMh iJtucly: 

1. The best way to devaLop reading ability is to provide abundant 
oiJjxHtunity for experiencing reading; 



V3 

4 



2. Student reading achievement is a direct function of what students do 
and how much of it they do; 

3. Teacher instructional approaches are more strongly and directly 
related to vrtiat students do and how much of it they do than tliey are 
related to reading achievement. 

Based on these expectations, the following typotheses were targeted for 
testing : 

Hi: The more time a student is allocated and attends reading instruction, 
the greater the achievement gain. 

H2: rtie greater the student's tijtie-on-task, the greater the achievanent 
gain. 

H3: The greater the proportion of student's time that is spent in 

task-related interaction with the teacher, the greater the achievement 
gains. 

H4: The greater the proportion of a student's time that is spent in direct 
reading activities, the greater the achievement gain. 

H5: Hie greater the proportion of a student's time that is spent in 
task-related interaction with the teacher, the greater the 
time-on- task. 

In order to test these hypotheses, measures of the amounts of instructional 
time spent in student behavior activities and teacher instructional approaches 
as well as other structural and contextual characteristics of Chapter 1 
programs were obtained. 



I NSIHUMENTftTIDN 
Instructional Time Relating to Student Activities 

Ihe qualitative aspects of reading-related activities that occur in the 
Chapter 1 classrooms have recently become the focus of much inquiry. 
Replicating Leinhart, Zignond and Cooley (1981) , the Iowa study examined the 
amount of tine devoted to activities that directly relate to the reading task 
in that they involve the student responding to print, and those that indirectly 
support sane aspect of reading. It is generally agreed that contextual reading 
provides the practice that is needed for the developnent of fluent reading 
skills (Allington, 1977; Shiith, 1978). Poor readers spend very little time in 
contextual reading (Allington, 1977; Gambrell, Wilson, Gantt, 1981) . Leinhart, 
Zignond, and CooLey (1981) reported that studfents who are engaged in more 
contextual silent reading made greater achievement gains; they failed to find a 
correlation of oral reading to achievement gains, in the Iowa study, direct 
reading activities were categorized as time spent in oral and silent reading, 
while the categories of phonics, canprehension, vocabulary, study skills and 
language activities were defined as the indirect reading behaviors. Itest 
taking was also included as an indirect reading activity given the amount of 
on-going diacpostic prescriptive teaching that occurs in Chapter 1 classrocms. 




In addition to the above mentioned student on-task categoriesr eight 
off-task categories were also defined for the Iowa study. These off-task 
categories included managementr waiting for further direction f ran the teacher, 
being disciplined, being academically unoccupied, working on academic 
assignments other than reading, and other off-task activities. Completing the 
off- task categories were two attendance related areas: being absent, and being 
in school, but not attending Chapter 1 scheduled class. Figure 2 sunmarizes 
the categories of student activities designated in this study. 



FIGURE 2 
STUDENT ACriVITy CATEGORIES* 



On-TSisk Reading Activities 

Oral reading 
Silent reading 
Canprehension 
Vocabulary 
Hionics 
Language 
Study skills 
Test taking 



Off-Task Activities 



Management 

Waiting for the teacher 
Being disciplined 
Academically unoccupied 
Working on other academic 

subjects 
Other off task activities 
Absent from school 
In school, but not 

attending Chapter 1 

class 



Actual Chapter 1 class time data were collected by the Chapter 1 teachers 
who kept individual daily attendance records for each student in the study 
(Appendix G) . The daily nunber of minutes were sumned to arrive at a total 
nunnber of minutes of instruction. These minutes were then converted to hours, 
which is the unit used to describe this variable in this report. 

Instructional Time Relating to Teacher Interaction and Kinds of Instruction 

Teacher interaction and kinds of instruction were measured utilizing 
Bloom's (1976) component characteristics of instruction: cueing, 
reinforcement, and corrective feedback. Blocm also included participation as 
an instructional characteristic. Within the design of this study, 
participation, or time-on-task, was included as a dependent variable of 
interest in its cwn ri^t, as well as an intervening variable explaining 
variation on achievement. Reinforcement and corrective feedback were canbined 
to form a category classified as positive feedback and one classified as 
negative feedback. It was necessary to include all logically possible forms of 
teacher interaction in order to (±tain a quantitative measure of total teacher 
interaction; thus. Bloom's categories were augmented with other possible forms 
of interaction, including a presentation category and a questioning category* 
Finally, since teachers spend time on-task with regard to reading instruction, 
but in a non-verbal form (namely, ty monitoring) , this category was likewise 



*Def initions for each of these categories can be found in Appendix B, Iowa 
Chapter 1 Reading Study Ctoserver Handbook . 
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added. These six kinds' of instructional interaction (cueing, presenting, 
positive feedback, negative feedback, questioning, and monitoring) conbined to 
form the measure of both teacher task-related activities and kinds of 
instruction. 



Teacher interactive behaviors with respect to reading instruction were also 
delineated on an off- task dimension. The amount of time teachers were off- task 
yet interacting with students was observed, and was categorized according to 
the source of the off-task bdiavior. Disciplinirj stu^nts, management of 
class activity, and other off-task activities (incluo^ng building rapport, 
giving rewards, etc.) , became the interactive off-task teacher behaviors for 
this stud^. 

A final teacher behavior category of non-interaction was included. This 
reflected times when the teacher was not actively involved with students. 
Figure 3 belcw suranarizes the teacher instructional approaches designated in 
this study. 



FIGURE 3 

TEACHER INSTFDCTICNMj APPROACH CRTBGORIES* 



Interactive On-Task Interactive Off-Task HorKEnteractive 

Cueing Management 

Presenting Di scipl ining 

Monitoring other off-task activities 

Questioning 

Providing positive 

feedback 
Providing negative 

feedback 



Structural and Contextual Chapter 1 Program Characteristics 

In addition to the student activity and teacher instructional approach 
variables already described, data were also collected in the Iowa study on 
other general Chapter 1 program characteristics. Through an observer-conducted 
interview with the regular classroan teachers, descriptive data were obtained 
regarding regular classroans from which the Chapter 1 students 
came. Figure 4 lists the additional general Chapter 1 variables on which data 
were collected, and Figure 5 lists the regular classroom variables on which 
data were collected. 



*Def initions for each of these categories can be found in Appendix B, Iowa 
Chapter 1 Reading Study Observer Handbook . 




FIGURE 4 
GENERAL CHAPTER 1 VARIABLES* 



Group size 
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not attending Chapter 1 class 
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Allocated Chapter 1 class time 


placement 


Actual Chapter 1 class time 


Pretest scores 
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Actual days of instruction 


Gains 
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REGULAR CLASSROOM VARIABLES* 


Allocated total reading time 


Allocated group reading time 


Class size 




Group size 





teacher's certification 



Site Selection 

In this study, a site was defined as a grade within a building within a 
district; thus it was possible for one district to generate multiple sites. It 
followed then, that a site could have several groups within a grade. Groups 
were defined to be the smallest unit of instruction where all students were 
taught at the same time. For exarrple- if grade two had 15 Chapter 1 students 
in the program, and these students were divided into three units, the units 
were defined as groups. 

Chapter 1 reading programs in grades two and four were the focus of this 
study. The selection of these grades allowed for representation of both lower 
and upper elementary classes where the majority of Chapter 1 funds are 
concentrated in the state of Iowa. In order to ensure variation on achievement 
among sites selected, the population of Chapter 1 classes was stratified on the 
basis of consistent low, average or high achievement. The criteria used in 



*Conplete definitions of these variables can be found in i^^ndix 
Definitions of General Chapter 1 and Regular Classroom Variables. 
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stratifying classes were as follows: 



1. The high achieving sites had demonstrated gains of at least twelve 
Normal Curve Equivalents (NCEs) for the preceding two years; 

2. The average achieving classes had demonstrated gains ranging from one 
to twelve NCEs for the preceding two years. 

3. The lower achieving classes had demonstrated gains of less than one 
NCE for the preceding two years. 

The intent of selecting from each of these three stratification cells was 
to maximize variation on achievement. If selected classroom variables do make 
a difference in achievement gains, these relationships can be detected by 
selecting classes varying in the size of achievement gains. Evaluation results 
from two preceding years were employed in assigning sites to the three 
categories of achievement in the attempt to improve the validity of the 
distinctions. Of the 105 sites ultimately selected for participation in this 
project, all nad demonstrated this consistency over two years performance, and 
more than 80 percent had demonstrated this consistency over three years. This 
finding provided additional support for the beliei lat the stratification 
procedures employed would yield a sample that provided variation on 
achievement. 

One hundred seventy-two sites from 89 districts met the criteria for 
inclusion in the study's sanpling frame. . Class schedules were requested from 
these potential sites; a total of 419 classes were identified as the basis for 
selecting instructional groupings on which observations could be conducted. An 
extensive analysis of these class schedules was done to identify potential 
sites that were geographically proximate in order to maximize the utilization 
of observer time in the field, and to generate the largest possible number of 
instructional groupings that could be observed given budget constraints. One 
hundred five sites from 64 districts were selected for observation, including 
93 second grade groups and 93 fourth grade groups. These groups generated 699 
participants, 604 of whom had valid scores at the end of the project and could 
be included in the analysis sample. 

The distribution among the three stratification cells of the sites 
ultimately selected for inclusion in the project is shown in Table 1. 



TABLE 1 

SITE SELECTION BY ACHIEVEMENT LEVEL AND GKADE 



Achievement Level 



Grade 2 



Grade 4 



Low Achievement 
Middle Achievement 
High Achievement 



21% 
38% 
41% 



27% 
54% 
19% 
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student Selection 



All of the participating sites followed the constraints associated with the 
(Jiapter 1 Evaluation Model, A~l. Even thou<^ nine different test batteries 
were used by the sites, in all but four cases either the comprehension subtest 
or total reading score was reported. Hierefore, these scores were considered 
to be based on tests that were comparable in content. Because the majority of 
Chapter 1 reading programs in the state of Iowa are pull-out programs , (where 
the instructional approach is diagiostic-prescriptive and the instruction 
occurs in a location other than the regular classroom) only students from tljese 
programs were included in this study. A^^ndix D contains a more detailed 
discussion of the student selection procedure and student achievement data. 

Chapter 1 Treacher Orientation 

Informational meetings were held prior to the beginning of the data 
cdlection process for all local Qiapter 1 staff who were involved in the 
project. The purpose of the meetings was to ejqplain the project and answer 
questions and concerns that mi^t cause anxieties among the teachers involved. 
Appendix E details the orientation process. 

Data Collection 

Data for this stud/ were collected by trained onrsite observers* who 
observed and recorded the activities in the selected groups for five days at 
three points in time between pretesting and posttesting. On each of the 15 
observation d^Sf students were observed 15 times for 15 seconds. Bie nature 
of student-teadier interaction was recorded for eacti student observation. 
Teachers were additionally observed for 15 fifteen-second intervals. A 
complete discussion of the observation process can be found in the manual r Iowa 
Chapter 1 Reading Study Observer Handbook f which is included in Apendix B. 

Data were gathered utilizing four different collection forms. A brief 
description of each is given below, and examples of each can je found in 
Appendix G. 

1. Iowa Qiapter 1 Reading Stu^ (X)6ervation Record The form used by 
observers to record individual student activities and teacher 
instructional approaches. 

2. Iowa Chapter 1 Reading Study Pretest-Posttest Information Pomi 
Completed ty the dapter 1 instructors in the study, this form 
provided pretest and posttest information relative to test batter/, 
form, level and subtest used, as well as individual pretest and 
posttest scores. Colimns for minutes and days of instruction were 
also included on this form. This information was conoputed by the 
observers based on information from the Iowa Chapter 1 Reading Study 
Student Attendance Record. 



*See Appendix F for a complete discussion of the selection and training of 
observers. 
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3 . Iwa Chapter 1 Reading Stud/ Student Attendanoe Recx)rd This form was 
an individual daily attendance form kept by the Chapter 1 teadiers for 
each of the students in the project. Actual instructional time was 
obtained fran this data. 

4. Icwa Chapter 1 Reading Stud^ Qassroan/General Chapter 1 Information . 
This form was used ty the observers to gather information on selected 
regular classrocmf Chapter 1 classrocmf student and teacher variables. 

All of these forms were hand-edited before data were entered into the 
computer. Additional quality control measures were applied when all of these 
separate data f H.es were merged into one file and a cross check (of 
identification information) was run. In addition, all ranges of the frequency 
distributions were verified for reasonableness. 



This section contains descriptive information gleaned fran the data 
collected in this study. While this information may be considered interesting, 
the reader is cautioned to remember that these findings are not to be 
interpreted as what is ri^t or best. These are simply statements of what is 
happening in Iowa Chapter 1 reading projects. 

Student Utilization of Time in Chapter 1 Reading 

Tiables 2 and 3 describe hew Chapter 1 students spent their time in specific 
activities. The average percent of time is given, as well as a conversion of 
that percent to an average actual amount of time in hours and minutes. These 
averages are for the time between pretesting and posttesting only aiKl thus do 
not describe the amount of instruction received within a school year. 

For both second and fourth grades. Chapter 1 students on the average spent 
about three-fourths of their class time in on-task reading activities. Chapter 
1 groups varied widely in this regard, with task engagement ranging as low as 
30 percent in fourth grade and as high as about 95 percent in both grades. 

Conversely then, approximately 25 percent of Chapter 1 class time was spent 
in off-task activities. The biggest usurper of off-task activities was 
management activities, which accounted on the average for about 12-14 percent 
of class time. Management activities appeared to require sanevrtiat more time in 
second-grade classes than in fourth-grade classes. The next largest factor 
accounting for off-task activities for both second graders and fourth graders 
was observed to be the fact that students were acadKnically unoccupied (e.g., 
disrupting, daydreaming, etc.). This accounted for about 4 percent cf the 
students' available class time. 
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Ti\BLE 2 

DESCRIPTICN OF HCW SECOND GRADE 
CHAPTER 1 READING SIUEENTS SFENT CLASS TINE 



Average Average 
Student Activities Percent Of Time Amount of Actual Time 



On-Task Reading Activities 



Phonics 


24 


12 


hours 






Conpsr ehen si on 


1/ 


8 


hours 


20 


minutes 


VOCaDUlaiy 


1 A 

14 


7 


hours 






Language 


10 


5 


hours 






Silent Reac3ing 


4 


1 


hour 


55 


minutes 


Oral Reading 


4 


1 


hour 


55 


minutes 


Study Skills 


<1 






10 


minutes 


Test Taking 


<1 






25 


minutes 


SIBTDTAL 


75 


37 hours 


15 


minutes 


Off-Task Activities: 












Management Activities 


14 


7 


hours 






Academically Unoccupied 


4 


2 


hours 


10 


minutes 


Other Off-Task Activities 


3 


1 


hour 


55 


minutes 


Waiting for Teacher 


2 


1 


hour 


20 


minutes 


Being Disciplined 


<1 






5 


minutes 


Other Academic Subjects 


<1 






10 


minutes 


SIBTOTAL 


15 


12 hours 


40 


minutes 



ERIC 



21 

12 



TABLE 3 



D ESQRIPTK^ OF HCW FOJRTH GRADE 
CHAPTER 1 READING STOEENTS SPENT CLASS Tim 



A/erage Mean 
Student Activities Jr^eroent Of Time Amount of Ac±ual Time 



On-Task Reading Activities 










Comprehension 


27 


12 hours 


40 


minutes 


Hionics 


14 


6 hours 


25 


minutes 


Vocabulary 


14 


6 hours 


55 


minutes 


Language 


8 


^ iaj lu. o 






Silent Reading 


4 


^ ilULU. o 


10 


minutes 


Oral Reading 


3 


± iKJUL 


30 minutes 


Study Skills 


5 


^ nouL s 


minutes 


Test Taking 


2 




45 


minutes 


SIBTOTAL 


77 


36 hours 


45 


minutes 


Qff~Task Activities: 










Management Activities 


12 


5 hours 


40 


minutes 


Academically Unoccupied 


4 


1 hour 


55 


minutes 


Other Off-Task Activities 


3 


1 hour 


35 


minutes 


Waiting for Teacher 


2 




50 


minutes 


Other Academic Subjects 


1 




40 


minutes 


Being Disciplined 


<1 




3 


minutes 


SIBTOTAL 


23 


10 hours 


40 


lUnutes 
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other off-task activities (such as building report, receiving rewards, 
selecting books to take home, celebrating birthdays, etc*)r accounted for 
another 3 percent of off-task activities. In both second and fourth grades, 
less than 1 percent of the students" time was spent in being disciplined. 
Fourth grade Chapter 1 students were more likely to be off-task with respect to 
reading but doing other academic subjects. However, the typical amount of time 
thus spent wcis small. In both grades, students spent about 2 percent of their 
time waiting for tSieir teacher for further direction. 

Although both second and fourth graders spent about the same proportion of 
time in reading-related activities, the relative enphasis placed on activities 
differed between the grades. Major differences occurred in the amounts of 
emjdiasis placed on phonics and coirprehension. On the average, 24 percent of 
second graders" time was spent in phonics activities versus 14 percent of time 
for fourth graders; 27 percent of fourth graders' time was spent in 
cQirprehensio' activities, versus 17 percent for second graders. 

Hie third most frequent activity for both second and fourth graders was 
vocabulary, ^proximately 14 percent of available time was spent on vocabulary 
in both grades. Language activities accounted for another 8-10 percent of 
time, and silent and oral reading for approximately 8 percent. Chapter 1 
second graders spent relatively little time in study skills activities or 
test-taking (less than 1 percent each) ; fourth graders spent about 7 percent of 
time in these activities. 

In summary, phonics was the single most frequent task activity for second 
graders, and the second most frequent activity for fourth graders. In grade 
four, the most frequent tasl: activity was conprehension^ Comprehension was the 
second most frequent second grade on- task activity. Other than these two 
variables, all other activities were fairfv similar in both grades. 

No differences were found between grade two and grade four in regard to the 
amount of time scheduled for Chapter 1 instruction. In both grades, the 
average class period for Chapter 1 instruction was approximately 30 minutes. 
The range was 15 minutes to 40 minutes for both grades. 

The average number of hours of actual instruction between pretesting and 
posttesting in grade two was 49 hours, ranging from 8 to 112 hours. Hie 
instructional exposure for grade four was very similar; the average was 45 
hours, and the range was from 10 to 73 hours. 

In grade two, an average of 112 days was available between pretesting and 
posttesting. Of those 112 days, students attended and received Chapter 1 
instruction for 97 days.* Hiey were absent an average of 5 days. Chapter 1 
students were in school, but did not attend Chapter 1 classes, for an average 
of 10 days. Thus, Chapter 1 students lost about 10 percent of the available 
Chapter 1 instructional time through non-attendance even when in school. 



*Iowa law requires a minimum of 180 days of instruction in school year. The 
average number of days of instruction between pretesting and posttesting in 
this study was considerably fewer than this given that enpirical norm dates .^or 
testing occur in Octctoer and May. 




In fourth grade, an average of 114 days of instruction was possible between 
pretesting and posttestirig. Fourth graders were absent an average of four days 
(one day less than second graders) . However, there were 14 days when fourth 
graders were in school, but did not attend Chapter 1 scheduled class. This is 
four more days than the second graders missed, and amounts to 12 percent of the 
days available for instruction. Table 4 details this attendance information. 



TABLE 4 

AVERAGE SCHEDULED AND ATTENDED DAXS 
CF CHAPTER 1 CLASSES BY 
CHAPTER 1 READING STODENTS 





^na 
Averaqe 


Grade 

Standard 
Deviation 


4th Grade 

Standard 
Averaqe Deviation 


Scheduled instructional days 
between pretest and posttest 


112 


19 


114 22 


Number of days Chapter 1 
students attended scheduled 
instruction 


97 


8 


97 7 


Number of days Chapter 1 
students were in school, but 
did not attend Chapter 1 classes 


10 


9 


14 13 


Number of days Chapter 1 
students were absent from 
school 


5 


3 


4 3 



Correlational analysis demonstrated that students participating in the 
Chapter 1 program receive different treatment, depending on their entrance 
level into the program. Students with higher pretest scores ^pear to be 
treated differently than those with lower pretest scores. Even though some o 
the correlations are not particularly strong, they do demonstrate different 
activity profiles for these two types of students (^^ppendix H, Table A-I) . 

In second grade, students with higher pretest scores spent more time in 
comprehension activities (r = .14) and language activities (r = .15) . ihey 
were also found to spend more time taking tests (r = .16) , perhaps indicating 
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that their progress was monitored more frequently than stuc3ents entering the 
program with lower scores. In addition, these higher pretesting students were 
acadonically unoccupied more of the time that they were in class (r = .13) . 
Tfeachers spent more time presenting to these students (r = .14) and also gave 
them more cues (r = .14) . Teachers were also interactively off-task with these 
students more than they were with lower achieving students (r = .16). 

On the other hand, students with lower pretest scores spent more time on 
phonics activities (r = -.17). They waited more for further directions from 
the teacher (r = -.19) and worked on other academic subjects more as well 
(r = -.28). These students were also involved in other off-task activities 
more (r = -.17). Teachers gave more negative feedback to these students than 
they did those who scored higher (r = -.22). 

In fourth grade the differences were similar. The students with hi.gher 
pretest scores spent more time in comprehension activities (r = .18), and 
management appeared to be important for both them (r = .16) and the teacher 
(r = .13). As in second grade, teachers presented more (r = .14) and provided 
more cues to these students (r = .14). 

Also, at the fourth grade level, the students with lower pretest scores 
were involved in programs that had larger caseloads (r = -.21). in addition to 
spending more time in j*ionics activities (r = -.13), they were also 
academically unoccupied more (r = --.11) and received more discipline (r = -.33) 
from their teacher, than did the higher pretesting students. 

Chapter 1 Teacher Utilization of Time 

In the manner used to illustrate Chapter 1 students use of time. Tables 5 
and 6 describe hew Chapter 1 teachers use their time. 

kjU the average. Chapter 1 teachers were engagaged in reading-related 
instructional activities in about the same proportion that Chapter 1 students 
were task'-engaged (approximately three-fourths of their class time) . One 
notable, although not large difference was the fact that fourth grade students 
tended to be slightly more task-engaged than second graders (77 percent versus 
75 percent) ; fourth-grade teachers, however, were somewhat less engaged in 
instructional reading activiti.es than were second grade teachers (70 percent 
versus 75 percent). 

When working with second graders, teachers spent about one-third of their 
time presenting; when working with fourth graders, they spent about one-fourth 
of their time in this activity. Monitoring constituted the single most 
frequent instructional activity for fourth grade teachers; it was the second 
most frequent activity for second grade teachers. 



TABLE 5 

DESCRIPTION OP HOW 
SBCCM) GRADE CHAPTER 1 
TEACHERS SPENT CLASS TIME 



Teacher Interaction 



Average 
Percent Of Time 



On-Task Instructional Activities 

Presenting 31 

Monitoring 21 

Questioning 19 

Cueing 2 

Providing Positive Feedback 1 

Providing Negative Feedback 1 

SUBTOTAL 75 



Average 
Amount Of Actual Time 



15 hours 
10 hours 
9 hours 
1 hour 



10 minutes 
20 minutes 
20 minutes 

30 minutes 
30 minutes 



36 hours 50 minutes 



Off -Task Instructional Activities 

Management 14 

Non-Interactive 7 

Other Off-Task 3 

Disciplining 1 

SUBTOTAL 25 



6 hours 
3 hours 
1 hours 



55 minutes 
25 minutes 
30 minutes 
30 minutes 



12 hours 20 minutes 



EKLC 



26 

17 



ni5x:RiPTioN OF now 
I'Xinmi qiapter i 

TI'VU:iIt'»RS SPEM* CLASS TIME 



TX^KiK^t Intrri^rt^ 

^ ^-Tank Instru ctio nal /k:tivitioL> 

Monitor ing 
Presentincj 
^i^jitionintj 
Cuein^; 

Pr(»vidin<j Positive FeedLvKk 
Ptovuimj ^kHJativt* Feedback 

' ^ t -Tdi;k Instiuetional Acti v ities 

Non-" Interact iv»^ 
Mana^iunent 
Otlier Ott^ ' jsk 



Aver aye 
Percent 01 Tjitk? 



28 
24 
14 

2 
1 

70 



14 

13 
3 

<1 



Average 
Amount Of Actual Time 



12 hours 
10 hours 
6 hours 



6 hours 
5 hours 
1 hour 



35 minutes 
45 minutes 
15 minutes 
55 minutes 
30 minutes 
30 minutes 



31 hours 30 minutes 



15 minutes 
55 minutes 
20 minutes 
30 minutes 



30 



14 hours 



^7 

la 



structural and Contextual Chapter 1 Program Characteristics 



On the average, Qiapter 1 instructional groupings in both second and fourth 
grades contained four students. In addition, the range tended to be similar in 
both grades: second grade groups ranged in si^e from 1-8 students while fourth 
grade groups ranged in size from 1-9 students. 

The averagp caseload for the 102 Qiapter 1 teachers in the study was 34 
students. The range, which spanned from 8-73 students, was souewhat misleading 
because some teachers were not full-time onpLcjyees. Those part-time Chapter 1 
teachers account for the Icwer portion of the range. On the other hand, some 
full-time teachers also had an aide working with than. In those instances, 
their caseload was larger. Sixty-ei^t percent of the teachers in the study 
had caseloads between 21 and 40, while 14 percent had frcHn 41-50 students; only 
5 percent had over 50 students. 

Data were gathered relevant to the number of prior years the obsetved 
Chapter 1 teachers had tau^t in a Chapter 1 program. The average number of 
years a teacher in the study had tau^t in a Chapter 1 program was ei^t. Ten 
percent of the teachers in the study tau^t in a Chapter 1 program for the 
first time during the 1983-84 school year, while another 5 percent were in 
their second year of Chapter 1 program teaching. 

The majority (67 percent) of the observed Chapter 1 teachers held a 
bachelor's degree, while 27 percent held a master's degree. The ranaining 6 
percent held a preprofessional certificate which indicates two years or more of 
training, but not the completion cf the bachelor's degree. 

The state of Icwa certification has a "reading approval" that can be 
obtained by ccmpLeting twenty hours of reading and/or reading related college 
courses. This is not a state reguironent to teach in a Chapter 1 program. 
However, 52 percent of the Chapter 1 staff in the study had the reading 
approval in addition to their other certification. 

Of the 699 students included in the study, 76 percent were selected using a 
norm- referenced test. The Icwa Tests of Basic Skills was the most oc«nmonly 
used instrument in that it was used 70 percent of the time. TabLe 1 describes 
the selection tests used for students in this study. An additional 17 percent 
of the students in the study were selected for Chapter 1 programs through a 
criterion- reference method as described earlier. 
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TABLE 7 



NQRM-REFiRENCED TESTS USED 
TO SHjECT STOEENTS fOR CHAPTER 1 PROGRAMS 



Instrunent 


Percent 


Iowa Tests of Basic Skills 


70 


Gates Mac Ginitie 


13 


Metropolitan Achievanent 


8 


Stanford Achievanent 


4 


Woodcock Reading Mastery Tests 


2 


California Achievement 


1.5 


SRA Achievement 


.9 


Metropolitian Readiness 


.6 



Seventy-four percent of the second graders were selected for Chapter 1 
participation using a norm- referenced test. The average entry level score for 
those students was 36.2 NCEs or the 26th percentile. In grade four, 91 percent 
of the students were selected using a norm- referenced test. Their average 
entry level score was 28.8 NCEs, or the 16th percentile. 

One-fourth of the second grade students were selected throu^ a procedure 
other than a norm-referenced test. The remaining three-fourths of the second 
graders who were selected with a norm- referenced test score had a mean entry 
level of the 26th percentile. This statistic, far below the 40th percentile 
allowedf demonstrated that Chapter 1 staff were correctly selecting students 
for the program following the legislative mandate of serving those with the 
greatest need first. This fact is even more evident in the fourth grade when 
almost all the students (91 percent) were selected with a norm-referenced testf 
and the mean entry level was even lower , the 16 th percentile. 

l&hle 8 is a sutiuary of the evaluation data generated ty the students in 
this study. It indicates average pretest data in terms of the mean NCE score 
of students in the study and the equivalent percentile of that average NCE. 
The same information is given for the posttest. The gain is the difference 
between the pretest average and the posttest average. 



TABLE 8 

AVERSE ERETEST SCDRESf POSITEST SOJBES MD GAIN 





Pretest 


Posttest 






Mean NCE Percjentile 


Mean NCE 


Percjentile 


NCE Gain 


Grade 2 


36.2 25 


50.8 


51 


14.6 


Grade 4 


38.8 30 


45.8 


42 


7.0 
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It is interesting to note that fourth graders begin at a hi^er 
pretest level than do second graders and their gain is not as great. The 
hi^er gain exhibited in grade two and the lower gain in grade four is 
consistent with the trends seen in both state and national Qiapter 1 data. 

Within the scope of this study^ it was not possible to investigate 
beyond the Qiapter 1 program for factors that impact on reading 
achievenent. In an attempt to capture some descriptive information about 
the regular reading program that Qiapter 1 supplemented/ interviews were 
conducted with dassroan teachers from whan the Qiapter 1 study students 
were drawn to c*tain seme descriptive information regarding regular 
classroom instruction. 

Qiapter 1 students observed came from 153 different regular 
caassroonSf averaging 21 students in size^ and ranging from 11-31 
students. In grade two^ regular classrocm teachers spent fran 3.5 to 15 
hours per week in reading instruction^ averaging a little over 9 hours a 
week, ^proximately three hours per week was the minimun time allocated 
for all reading in fourth grade^ and the maximum was about 11 hours per 
week. Fourth graders received an average of 6 hours per week of reading 
instruction in their regular dassroans. 

Regular dassroom teachers scheduled their Qiapter 1 students for 
instruction in groups ranging in size from 2-13 students in second grade, 
and from 1-18 in fourth grade. Group size averaged seven students in 
grade two and ei^t students in grade four. More specific information was 
also obtained regarding the amount of time Qiapter 1 students received 
reading in smaller instructional groupings within their regular 
classrocros. In second grade, Qiapter 1 students received reading 
instruction in small instructional groupings for an average of 3 hours per 
week, ranging from 20 minutes to 8^5 hours. In the fourth grade, students 
receive such instructj.on for an average of 2.5 hours, and ranging from 50 
minutes to 5 hours per week. 

Ei^ty-four percent of the regular classroom teachers who had 
Qiapter 1 students involved in this stud/ had a bachelor's decree; 
thirteen percent a master's degree, and the remaining 3 percent the 
pceprofessional certification, which implies two or more years of training 
but no bachelor's degree. In addition, 44 percent of these classrocm 
instructors had the reading apprcval in their credentials. This approval 
was not mandatory for elementary teachers. 



MALYTICMi FINDINGS 

Analyses were conducted at both the group level (where variation in 
the average group achievement was the focus of analysis) , and at the 
individual student level. The rationale for conducting analyses at both 
the group and student level was that the e3?>lanation of variation that 
occurs on one level of analysis may not be identical to the explanation of 
variation at the other. Specifically^ individual student level variation 
on achievement is generally more highly subject to individualistic 
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influences and variation in achievement. Group level data "averages over" 
much of this individual level variation. Both analyses were of interest 
to the study conducted herein. Because one might expect that 
substantially different factors may explain variation in gain for second 
graders and fourth graders, analyses were conducted separately for each 
grade. 

A stepwise regression approach was used to identify the variables that 
were significant in explaining variation in achievement gains.* The 
relative importance of each variable in explaining variation in 
achievemen'^• gains was assessed with che chan^r^ in the multiple r2 
statistic — a measure of the increase in the proportion of explained 
variation that each variable contributes given the existence of the other 
variables in the analysis; and with standardized regression coetficients, 
which indicate the relative importance of variables included in the 
models. Achievement gains were detained by subtracting the pretest score 
from the posttest score. 

Regression analyses were conducted utilizing three categories of 
variables in order to assess their separate, then combined, influence on 
student achievement. These categorizations corresponded to general 
classes of variables commonly investigated in attempting to explain 
variation in achievement and were those that had been identified for 
investigation in this study: (1) student activities , including the 
relative amounts of time students spend in on-task and off-task 
activities; (2) instructional approaches of the teacher, including the 
relative amounts of timD teachers spent in instructional on-task 
interaction and off-task interaction or non-interaction with students; and 
(3) structural characteristics of instruction, including the years of 
Chapter 1 teaching experience of the teacher, numbers of students in a 
group, teacher's total caseload, attended days of instruction, and actual 
amounts of instruction. 

One of the hypotheses of this study was that the more time students 
spent in on-task activities, the greater the achievement gains would be. 
Regression analysis indicated that this was true for grade four, but was 
not true in second grade. This finding is in keeping with the specific 
variables found to be significant at the respective grade levels. These 
specific variables are addressed in detail in the discussion that follows. 



* The tolerance level for the addition or deletion of a variable was set 
at .15, a commonly- accepted level of tolerance. 
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Second Grade Results 



The largest proportion of variation on achievement gain scores is 
typically explained ty the average initial pretest score: the lower the 
initial performance level, the greater the average gain score. This is 
not so much a research finding as it is a statistical phenanena.* In this 
stud/, the average second grade pretest bTore explained 37 percent of the 
variation on gains scores (l^ble 9) . 



TABLE 9 

SIGNIflCflNT VARI/fiLES 
GROUP LEVEL ANALYSIS 
GRADE 2 



Standardized Unstandardized Additional 
Coefficient Coefficient Contribution to r2 



Pretest Score 


-.52 


- .61 


37 


Oral Reading 


-.23 


- .04 


5 


Chapter 1 Experience 


.30 


.78 


4 


Phonics 


-.21 


- .01 


1 


Monitoring 


.14 


.004 


2 


TeadiPT Management 


.14 


.009 


2 


Teacher Other Off-Task 


-.37 


- .03 


1 


Student Other Off-Task 


.34 


.03 


1 


Positive Feedback 


.20 


.13 


1 


Being Disciplined 


-.17 


- .19 


1 


Group Size 


-.10 


-1.08 


1 



Total r2 = .76 

Adjusted = .72 

a < .15, Stepwise Approach 



*Lcwer pretest scores have less "restriction of range" and can demonstrate 
greater gain scores. Additionally, students who tout exceptionally low on 
pretests are more likely to irooke the "regression effect", whereby students 
test Iwer than their "true" performance (due largely to measuronent error) who 
demonstrate relatively larger gains on subsequent testing. 
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student Activities , The most important instructional variable e55)laining 
differmoes in average group achievement gains was the amount of time students 
spent in oral reading activities: the more time spent in oral reading/ the 
smaller the average achievement gain, Similarlyf the more time students spent 
in phonics activities and the more frequently they were disciplined/ the 
analler was the average achievement gain. Althou^Ji it was esqpected that 
increased frequency of all off-task activities would diminish the average 
achievement gainSf this esqpectation was not born out. In fact/ one off-task 
variable: expending sane amount of time in interactional off-task activities 
(such as building rapport/ receiving rewards and celebrating birthdays)/ 
contributed positively and significantly to averaye achievennent gains. Giution 
must be exercised in interpreting this finding; clearly^ increasingly greater 
amounts of time in essentially off-task activities could diminish the size of 
achievement gains. Students who received more discipline also had Icwer 
achievement gaiins. 

Ite second grade results were identical at both the individual student and 
group levels of euialyses. However/ at the individual level/ an additional 
finding surfaced: the more time an individual student spent on vocabulary 
activities/ the greater the achievement gain (Tl&ble 10) . Hiis finding 
supports the notion that second graders require a broad-tosed sight vocabulary 
and understanding of word meanings before they can derive understanding from 
print in context. 



TABLE 10 



SIGNIFTCMIT VARIffiLES 
INDIVIDUMi LEVEL MIALYSIS 
GRADE 2 



Variable 



Standardized 
Coefficient 



Unstandardized 
Coefficient 



Additional 
Contribution to 



Pretest Score 
Oral Reading 
Chapter 1 Experience 
Student Other Off-OSisk 
Teacher Other Of f-rTask 
Teacher Management 
Phonics 
Monitoring 
Positive Feedback 
Being Disciplined 
Group Size 
Vocabulary 



.49 
.22 
.23 
.29 
.26 
.13 
.16 
.13 
.11 
.06 
.06 
.07 



.56 

.04 

.70 

.03 

.03 

.01 

.01 

.003 

.07 

.10 

.003 

.96 



35 
3 



<; 



Total = .62 

Adjusted - .60 

a < .15, stepwise approach 



Instructional Approaches ^ TVo teacher instructional aH?roaches proved 
significant in explaining variation in average class achievement gains* The 
more time teachers spent monitoring student progressr managing instructional 
activities, and providing positive feedback to students, the larger were the 
average achievement gains* Teachers who spent larger amounts of their class 
time in interactive off-task activities with their students tended to have 
lower av^erage achievement gains* 

Structural Characteristics * One structural characteristic of Chapter 1 
instruction proved sigiif icant in explaining differences in the size of 
achievement gain: the larger the size of Qiapter 1 instructional group, the 
smaller the average gain. Also, the more years of e:q)erienoe teaching in a 
Chapter 1 program a teacher reported, the larger were his/her class average 
achievement gains* 

Second graders were on-task when teachers monitored instructional 
activities more frequently (r = *60) , when th^ received more feedback from 
teachers, either positive (r = *42) or negative (r = *45) , and when teachers 
provided greatei management of the instructional activities (r = *33)* 
Somewhat lower but iiKxaerate relationships were revealed between Chapter 1 
students propensity to be on-task and selected instructional approaches* 
Students were cteerved to be on-task when teachers interacted more frequently 
in task activities in general, as well as in terms of specific task 
interaction, presenting (r = *28) , cueing (r = *28) and questioning (r = *30) * 

Certain student task activities were also related to the students 
propensity to be on-task* In particular, a strong relationship was 
demonstrated between the amount of time students spent in phonics activities 
and on-task behavior (r = *60) , low-to-moderate relationships were demonstrated 
beta^een oral reading activities (r = *26), language activities (r = *32) and 
on-task behavior* Althou^ larger amounts of time devoted to phonics and oral 
reading activities aK>ear to generate more on-task bAavior, the reader is 
cautioned that hi^er frequency of these activities were negatively related to 
gains in achievement* The complete correlation table of actual amount of 
on-task time with student activities, instructional approaches and structural 
characteristics can be found in Appendix H, l&ble A-2* 

Fourth Grade Results 

Student activities and instructional characteristics were also investigated 
to identify important characteristics of reading instruction for fourth 
graders* Contrary to the typical e^ctation where the averagf initial j^etest 
score explains the majority of the variation on the gain score, pretest scores 
explained no more of t^ie variation than did the combined other in-class 
variables* This is a highly encouraging finding, since it provides support for 
the notions that thoughtful structuring of student activities, effective time 
management and the utilization of effective teaching practices can result in 
larger achievement gains for classes of all levels of initial performance* 
T&bles 11 and 12 report the significant fourth grade variables at the group and 
individual analysis levels respectively* 
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TABLE 11 



SIGNIFICANT VARIABLES 
GROUP LEVEL ANALYSIS 
GRADE 4 



Standardized Unstandardized Additional 
Variable Coefficient Coefficient Contribution to 



Vocabulary .36 .01 14 

Pretest Score -.30 -.26 10 

Chapter 1 Experience .28 ,59 6 

Oral Reading .22 .03 5 

Student Other Off-Task -.18 -.02 3 



Total = .38 

Adjusted r2 = .36 

a < .15, Steiptiise Approach 



TABLE 12 

SIGNIFICANT VARIABLES 
INDIVIDUAL LE7EI. ANALYSIS 
GRADE 4 



Standardised Unstandar jized Additional 
Variable Coefficient Coefficient Contribution to R?. 



Pretest Score 


-.47 


-.44 


18 


Vocabulary 


.13 


.01 


5 


Negative Feedback 


-.13 


-.06 


3 


Chapter 1 Ej^ierience 


.15 


.39 


2 


Being Disciplined 


-.13 


-.14 


1 


Student Other Off-Task 


-.15 


-.02 


2 


Language 


.11 


.01 


1 


Positive Feedback 


.09 


.06 


1 



Total p2 = .33 

Adjusted r2 = .31 

a < .15f Stepwise Approach 

Student Activities , The best single predictor of average group achievement 
gains in fourth grade was the amount of time students spent in vocabulary 
activities {defined as building basic si^t word vocabulary and word meaning) . 
The more time students spent in vocabulary activities, the larger was the 
average achievement gain. The more time students spent in oral reading 
activities, the greater was the average achievement gain. At the individual 
level of analysis, vocabulary was also the strongest predictor. In addition, 
the more time a student spent in language activities, the larger the 
achievement gain. 
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As lypothesized, several off-task activities were related to lower 
achievenent gains. The more time students spent in interactional off-task 
activities with the teacher (e.g. celebrating birthd^s, building rapport), the 
Icwer was the achievement gain. At the individual level of analysis, the more 
time (and the more frequently) the student was disciplined by a teacher, the 
Icwer was the achieveraent gain. 

Instructional AFProaches . At the individual student level of analysis, 
teacher instructional approaches proved significant in explaining achievenent 
gains. The more frequently teachers provided positive feedback to students and 
the less frequenUy they provided negative feedback, the larger was a student's 
achievement gains. 

Structural Characteristics . Similar to second grade findings, the more 
years of experience teaching in a Chapter 1 program a teacher reported, the 
larger were both average group and individual student achievenent gains. 

Additional correlational analyses were also conducted at the group level to 
identify characteristics of fourth grade reading instruction that were related 
to a students" propensity to be on- task or off-task during their Chapter 1 
instruction. All of the student on-task activities with the exceptions of oral 
reading (r = .17) and test taking (r = .11) were hi^y correlated with total 
on-task time. The strongest relationships were with language, (r = .52), 
vocabulary (r = .45) and oanprd:iension (r = .40) . In addition, the student 
off-task activities of waiting for further direction from the teacher (r = .26) 
and management activities (r = .49) were also correlated with the frequency of 
total on-task behavior. All of the teacher interactive on-task activities with 
the exception of negativi^ feedback (r = -.03) were also correlated with total 
on-task time. The instructional approach of management activities (r = .46) 
was significant as well (See Appendix H, Table A-3) 



DISQJSSION 

It must be clearly pointed out that the analyses that were conducted were 
designed to e^^lain wty Chapter 1 programs vary in the size of their 
achievement gains. The results of such an analysis alone should not be 
utilized to identify effective instructional activities. A pertinant exampie 
of an erroneous interpretation of findings is exemplified by the failure of the 
amount of time spent in oomprdiension activities to be significant in 
explaining fourth grade gains. Conprehension activities wore the most 
frequently occurring activity. In all probability, this consistency of 
emphasis on the relative importance of conprehension activities accounts for 
its failure to significantly es^Jlain differdhoe in achievement ipins. This is 
not to say, hcwever, that it is an uniirportant activity. 

There are at least two possible explanations for this failure. First, 
perhaps the conprehension related definition of vocabulary activities (building 
word meanings) was the important aspect of the vocabulary conponent at the 
fourth grade level. Thus, when vocabulary activities explained a large 
proportion of the variance on achievement gains, it was this "splinter" 
conprehension skill that was contributing to that finding. 
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Secondly f fourth grade students spent on the average r nore than 25 percent 
of all time available in canprehension acuivities* In all probability, this 
lack of much variation on the amount of time spent in comprehension activities 
accounts for its failure to significantly explain achievement gains. 
NonethelesSf ccmprdiension activities are clearly iirportant to iiiprcving 
reading abilityr as supported by the effective schools literature. 

The iicportcuioe of the quantity of instructional time on student achievement 
has been the subject of much theoretical discussion and empirical 
investigations (Bloonr 1974; Carrollr 1963; Wiley and Harnischfegerr 1974; 
BrophYf 1979; Denham and Liebermanr 1980; Frederick and Walberg, 1980; Good, 
1979; Rosenshiner 1979, 1980; Filty and Cahan, 1978; Blockr 1975; Barr, 1974; 
1975; Anderson, 1973; Sirotnik, 1982; Evertson, 1980; Powell and Dishaw, 1980; 
Stallingsr 1980 r 1975; Pfannenstiel and Sewellr 1980; Good and Beckerman, 1978; 
Arlin and Rothr 1978; Carroll and Spearitt 1977) . Nonetheless., analyses 
conducted for this study reveal that sinpLy increasing the quantity of 
instructional eaqposure is not sufficient to attain greater achievement gains. 
In factf the ejqpenditure of greater amounts of time in certain reading 
activities may actually diminish the size of achievement gains. Spending 
greater or lesser amounts of time in specific instructional and interactive 
activities prcved to be the important aspect of instructional time. 

The general finding that not only the quantitative but also the qualitative 
aspects of instruction are important is in keeping with recent investigationsr 
althou^ the amount of research that has been conducted to support this notion 
has not been as extensive (Anderson, 1981; Centra and Potter, 1980; Karweit, 
1983; McDonald and Elias, 1976; Duffy, 1981; Slavin, 1983) . Both the amount 
and quality of oral reading activities has been demonstrated in past research 
to be iirportant aspects of the acquisition of reading skills. Providing 
opportunity for practice in the direct reading of print has been found to be 
important for the development of fluent reading skills. In regular classrooms, 
it has been found that low performing children spend as much as 90 percent of 
their time in oral reading activities (Allington, 1982) . One of the most 
striking findings of the present study was the canparatively aaall amounts of 
time second grade students in Oiapter 1 classes spent in either silent or oral 
reading activities (averaging about 8 percent of class time for both silent and 
oral reading) . 

The amount of oral reading was sigiif icant in esqplaining gains in 
achievement in both second and fourth grades. However, contrary to 
e3q)ectations of past research, oral reading was found to be negatively related 
to achievement gains in second grade. At least two possible explamtions for 
this negative relationship exist. Perh^ Chapter 1 second graders do not have 
the required skills to read in context. If this is the case, the acquisition 
of broad-based sight vocabulary and the synthesis of reading skills required to 
derive meaning f ran print mis^t be more appropriate ways to spend time. 
Secondly, researdi in regular classrooms suggests that silent reading should 
precede oral reading activities in order to prwide the practice necessary to 
acquire comprdiension (Samuels, 1985; Brecht, 1977; Durkin, 1983). Oral 
reading approaches vrtiere students have not been prcvided the opportunity tv 
practice are particularly detrimental to low performing children, providing 
them increased opportunities to hear other children (and themselves) stunbling 
over words and ther^ decreasing the level of ocmpr^ension. 
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Contrary to second grade f indingsr fourth grade results revealed that 
greater amounts oE time spent in oral reading activities increased average 
achievement gains. The amount of tiine spent in silent readingr howeverr was 
unrelated to achievement gains. One plausible explanation for these findings 
is that direct oral reading activities may be particularly effective for 
Chapter 1 fourth grade students if they allow for more individualized 
instructionr more corrective feedbackr and demand more concentration f ran the 
student than is true for silent reading activities. 

Second grade analyses further indicated a negative relationship between the 
amount of time spent on phonics activities and achievement gains. Qearlyr one 
should not interpret this negative relationship to indicate that time spent in 
phonics activities is totally inappropriate. Againr it is the amount of time 
spent in phonics activities relative to the e:?)enditure of time in other 
reading activities that is at issue. Phonics instruction has in the past been 
criticized for its detraction frcm "meaning" activities (Goodnanr 1976; Sraithr 
1973) , which has been attributed to "the false dichotony between phonics and 
meaning that has dominated the field of reading for so many years" (Beocxaing A 
Nation of Readers r 1985: 42). This criticism appears appropriate for the 
interpretation of findings in this studyr since it was found that the more time 
groups spent on phonics activitiesr the less amount of time they spent on 
comprdiension activities (r = -33) . Perh^ second grade groups that are 
characterized by large expenditures of time in learning the isolated reading 
skills of phonifjs are not provided sufficient amounts of comprehension-related 
activities. Althou^ the study was not designed to allow for a recoiroendfeition 
of an "optimal" amount of time that is appropriate for phonics activities in 
Chapter 1 classesr it is dear that expending as much as 65 percent of class 
time on phonics di m i n ishes the size of average class achievement gains. This 
finding supports recent reccnroendations that phonics should be taught early and 
kept siit^aer and should go hand in hand with opportunities to identify words in 
a meaningful context (Becaning A Nation of Readers, 1985). 

A finding at the second grade level r that appeared to be contradictory to 
fourth grade findings was tliat greater amounts of teacher time spent in other 
off-task interactions with students tended to diminish the size of their class 
gains. Further investigation revealed that teachers who spent more cf their 
class time in other off-task activities (such as building rapport with the 
students) also tended to be characterized ty sane less-desirable instructional 
practices. For exampler teaciiers who spent greater amounts of time in other 
off-task interactions also tended to be noi>-interactive a great deal of the 
time (r = .55) ; they spent more time disciplining students (r » .55) ; and their 
students spent more time in other off-task activities (r « .41) as well as 
being acadanically unoccupied more frequently (r = .66) (Appendix E, T&bLe 
A-4) • 

Further support for the theoretical ijiportanoe of positive and supportive 
student-teadier interactions was provided ty the findings in both second and 
fourth grades that the prevision of more frequent positive feedback to students 
contributed to greater achievement gains; and the finding in the fourth grade 
that the prevision o£ more frequent negative feedback reduces the size of 
achievement gains. Additionally^ classes characterized ty teadiers who 
interacted more frequently with their students demonstrated a greater frequency 
of on-task bdiavior in both seooid an6 fourth grades (r ■ .85) . 
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Past research has demonstrated that positive and supportive interactions 
are important aspects of instruction, especially for younger, Icwer^achieving 
students (Brophy and Evertson, 1976; Cantrellr Stenner and Ketzemeirerf 1977; 
Good and Beckermanr 1978; Brophyr 1981). Olie amount of time students spent in 
other off-task activities was positively related to student gain. The 
interpretation of this finding must be tempered by the fact that second grade 
classes differed in the amounts of time spent in these activities in three 
ways: they tended to spend no amount of time in such activitieSf they spent an 
a7erage of 3 percent of their time in these activitieSf or they spent as much 
as 24 percent of their time thus engaged. Therefore/ the e^ipenditure of some 
amount of time in other off-task activities fcy students for the purpose of 
building rapport or providing rewards contributed significantly to student 
achievement. Hcweverr with sane of the observed classes spending up to 24 
percent of their time in such activities may well detract f rem the e3?)enditures 
of time in meaningful instructional activitieSf which is a major objective of 
the rapport-building activities. 

The effects of size of instructional grouping on adiievement has been the 
subject of much speculation and seme research (see Glass et al., 1982 for a 
review of research findings) . Generally, the anticipated negative effects of 
larger instructional groupings has only been demonstrated in regular classroon 
research where a large amount of variation in the size of instructioral 
groupings exists (e.g. between groups larger or smaller than about 30 
students) . Despite the fact that the average size of a Chapter 1 instructional 
group in second grade was only four students (and ranged f ran one to eight 
students) f the negative effect of larger group size was demonstrated. In order 
to identify the potential reasons wly group size negatively affected 
achieverent gains, correlational relationships were examined. They revealed 
that students in larger instructional groupings spent significantly larger 
amounts of their time being disciplined (r = .23) r waiting for further 
direction from the teacher (r = .22) and being academically unoccupied (r = 
.20) (Appendix H, Table A-5) . 

Bast research findings have demonstrated a positive relationship between 
the overall quantity cf teacher interactions and achievement gains (Jackson, 
1968) and the amount of on-task interactions and achievement gains (StallingSr 
Needles and Stayrockf 1979). Findings in this study further this research by 
specifying the types of interactions that are related to achievement gains. 
The amounts of time teadiers spent in specific instructional activities related 
to achievement gains. Teachers of the second grader who spent more time 
monitoring their students' progress and activities/ demonstrated larger 
achievement gains. This finding is consistent with recent evidence which 
indicates that in-class instructional strategies that are aimed at imnediately 
clarifying students' misoonoeptionSf and that provide opportunities to adapt 
instructional activities to meet students' interests and needSf are more 
beneficial for student learning (Wang and Lidvall, 1984; Shrcyerr 1978; 
ConnerSf 1978). Additionally/ teachers who spent adequate amounts of time in 
management activities apparently yielded learning envirorments that were better 
organized for instruction and demonstrated greater gains in achievement. 

A finding discussed earlier examined the amount of time students actually 
attended Qiapter 1 classes in comparison to how much time actually was 
available, i^roximately 10 percent more time was available than what students 
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actually attended. Through regression analysis, it is possible to 
theoretically project hew gains would be influenced if instructional time is 
increased, and the increased amount of time is spent in the identified 
activity. It is important to remanber that all other activities would have to 
be continued the same as was done in the past, and that the anphasis on a 
particular activity would have to be in addition to that time» 

Using the significant student and teacher b^aviors f ran the group level 
analysis, Tables A-6, and A-7 in Appendix H e:q)lain how much time a 10 percent 
time increase in a given activity would theoretically influence NCE gains. 

T^ble 6 states t' t in this study an average of 415 minutes was spent on 
vocabulary activities ty individual students in fourth grade between pretesting 
and posttesting. If this time is increased by 10 percent, an additional 42 
minutes would be spent in this activity. If this were done, the gain could be 
increased ty approximately .2 of one NCE. Similarly, decreasing the amount of 
time spent in other off-task activities by 10 percent, or 10 minutes could also 
increase gain by approximately .2 of an NCE. 

The difficulty in applying such findings to "real-life" Chapter 1 classes 
is that all of the other activities that have been carried on in the past, must 
continue to be carried on, and to the same degree as before. Then the change 
in time is in addition to or less than these continued activities. Only if the 
exact amounts of time indicated in T&bles A-6 and A-7 are devoted to the 
activities included, will the NCE changes indicated result. This information 
can be valuable to LEA personnel in that it establishes, in terms of actual 
time, some bench-marks to use in planning instructional activities. 

At both grades and at both the individual and group levels of analysis, the 
experience of the teacher consistently appeared as a significant variable. As 
discussed earlier, this study iiade no attempt to define what teacher experience 
is; the only information gathered was the number of years an individual had 
taught in Chapter 1 classes. However, this finding is one that adninistrators 
may wish to keep in mind when emplcying and assigiing staff to Chapter 1 
positions. 
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QUflLIFICATIDNS AND RESPONSIBILITIES OF 



DIRECTOR AND OIHER KEY PERSONNEL 



Dr. Oliver T, Hlmley y the project director, has been associated with 
Chapter 1 efforts since 1970 when he assuned the position of chief of the Iowa 
Chapter 1 Section. His interest in imprcving program quality at the LEA level 
has been evident in his years of working with Chapter 1 programs. Collecting 
defensible data and requiring stringent quality control of the processing have 
been a given for Dr. Himley. His intended utilization of sudi c^ta has been to 
provide information to LEAs and also present findings regarding factors that 
contribute to successful programs. This stud/ is a continuation of these 
efforts. 

Dr. Himley has been recognized at the state level for his contributions in 
the reading area. He was presented the 1983 Reading Ser/ice Award fran the 
Iowa Reading Association. In addition, he has been a visible figure at the 
national level having worked closely with Department of Education personnel as 
well as the offices of key Senators and Representatives. 

All budget concerns for this project were the responsibility of Dr. 
Himley. He was also the person responsible for all ultimate decisions 
regarding this project. 

Coleen McQanahan ^ assistant project director, has a comprehensive 
background in elementary reading and Chapter 1. Her teaching e3?)erience 
included preschool, first grade and Chapter 1 reading. This teaching 
experience linked with her adninistrative e:q)erience of directing a Chapter 1 
program that included a staff of twelve teachers and twelve teacher associates 
created a background highly relevant for the stu^. The Chapter 1 program 
which she coordinated was given National Joint Dissemination Review Eanel 
(JERP) approval six months after she left that position to become a Chapter 1 
consultant at the Iowa Department of Public InstructicMi (DPI) . 

Reviewing Chapter 1 programs for the DPI and continued involvement in 
professional organizations have continued to keep Ms. McQanahan abreast of 
what is happening at the local level. In her work at the State Chapter 1 
level, she has been the evaluation OMisultant, has reviewed programs and 
presented Workshops at the Area Education Agency (AEA) level for Chapter 1 
personnel. The content of these workshops has included evaluation 
requirements, e:5>lanation of Model ^V-1, e35)lanation of the NCE, use of 
evaluation feedback reports, appropriateness of functional level testing and 
considerations to be used for proper test selection. 

Ms. McQanahan was responsible for the day-to-day conduct of this project. 
Her major responsibilities included the designing of the observer training 
materials and sessions, overseeing the actual observations, directing the 
quality control of the ^ta, providing input into the analysis process and 
writing the reports. Beyond this, the additional planning and dissemination 
and technical assistance efforts will be a prime responsibility of hers. 
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Dr. LeLand Tack y chief of the Data Analysis and Statistics Section of the 
Icwa DPIr served as the principal analyst for this project. Dr. Tack has 
served as a technical advisor in the development of the Iowa Chapter 1 
evaluation systan. Much of the success of the quality control aspects of the 
system can be attributed to Dr. Tack. In addition to this, he has conducted 
all of the prior state- level investigations of the Qiapter 1 data. 

In his capacity as consultant, director and chief at the Iowa DPI, he has 
had extensive e3?)erience in applied statisticSf data processing and research 
methodology for both large and anall scale projects. He has been involved with 
instrunent developmentf testing, surveying, coding, analyzing and reporting 
data. In addition. Dr. Tack has been a consultant to other staff members with 
respect to research design, sampling, survey instrument design, ^ta gathering, 
data coding and analysis. 

Dr. David Alvord , consultant in the Data Analysis and Statistics Section of 
the Icwa DPI, served as assistant analyst for this project. Dr. Alvord brought 
experience fran the elenentaiy throu^ post-secondary levels to this project. 
He was involved in the initial design of the project. 

Ms. Judy Pfannenstiel , Assistant Project Director and Senior Research 
Associate with the Chapter 1 Ttechnical Assistance Center (TAG) , was called upon 
by the SEA for assistance in this project. Ms. Pfannenstiel was involved in 
the research design of the project 5 in the data analysis and report writing. 

Ms. Pfannenstiel has demonstrated expertise in project management, 
coordination, and integration. She also has significant e:q)erience in <feta 
analysis and is thoroughly familiar with the procedures and methods of both 
quantitative and qualitative analysis as employed in hi^ quality evaluations. 
In addition, Ms. Pfannenstiel is familiar with the difficulties inherent in 
conducting credible program evaluations at the local level. She is hi^y 
capable of integrating evaluation procedures with the ptrogramnnatic activities 
of local education agencies. 

Dr. Carol Vaoek , TAC Research Associate, provided input for the design of 
this projects Because she terminated her emplcyment with the TAC as of August 
1, 1983, she was not involved in the project beyond that time. Her background 
in the area of reading as well as Chapter 1 provided invaluable assistance to 
the SEA. 

Dr. Dianne Seltzer, TAC Senior Research Associate, continued the work on 
this project begun ty Dr. Vacek. The SEA involved her in the developnent of 
materials for observer training sessions, conduct of initial observer training 
sessions and follow-ip training sessions, data analysis and report writing. 

Dr. Seltzer brou^t expertise to this project in the area of reading. Her 
research and teaching e3?)erienoe at the elementary school as well as college 
level provided a broad base of experience to the project. 
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Background Information 

Chapter 1 of the Education Consolidation and Improvement Act of 1981 
provides financial assistance to meet special educational needs of 
children. Programs that are functioning under this legislation must be 
supplemental, which means that students who attend Chapter 1 classes also 
get as much reading instruction f ran their regular classroom teacher as 
all other students and then receive Chapter 1 instruction in addition. 
Generally Chapter 1 classes take pLaoe in a location away f ran the regular 
classroom. Also, instruction is individualized to the extent possible and 
therefore Chapter 1 students meet in small nroups. A variety of methods 
and materials are emEicyed in Chapter 1 programs. As a rule, not all 
students in a class do the same assigiment, therefore several activities 
can be on-going simultaneously. 

An observer can expect to find Chapter 1 classes meeting f ran 20-40 
minutes with from 1-8 students in the class. 

Chapter 1 is primarily the ^trie program that was known as "Title I" 
prior to 1981. The change in legislation changed the name of the program. 

Each year the students in Chapter 1 classes are given a pretest and a 
posttest as part of the evaluation i^ooess to ascertain the amount of 
achievement gain that has occurred during the school term. These data are 
collected annually at the state " ^el and are aggregated at grade, 
building, district and state leve^.s. In addition to pretest and posttest 
information, data ore a3 so collected on other program variables including 
caseload, program type, time in the program, and student- instructor 
ratio. Using these iaca, the lo Chapter 1 Section has implanented an 
evaluation system th,.t generate*- ni^ qiality data. However, only a anall 
amount of the achievTnent ^ in a be explained ty tne data collected on 
selected variables. Tb^s at least partly due to the fact that these 
global measures do not vary groatly throughout the state. Obviously, 
other factors are involved that contribute to successful Chapter 1 
programs. Determining what these other factors are is the subject of this 
study. It appears that the kind of data reqaired to identify those 
factors cannot be collected via Iowa's current evaluation documents. In 
addition, within-clasrircom data is needed. This stud/ will conduct an 
indepth examination of representative programs at the LEA level. In this 
manner, the differences in achie/onent will be determined ty identifying 
the amounts and kinds of within-class variables present. 

Chapter 1 Study Information 

The purpose of this study is to conduct an on-site study of 
within-class variables to determine those that affect achievement. The 
study will focus on reading programs in grades two and four. Observers 
will be assigned to specific "sites" to do their observations. A site is 
defined as a grade within a particular building in a given school 
district. Not all Qiapter 1 students in a grade will necessarily be 
observed. The nimber of groups observed will depend on scheduling. Only 
the grade specified at a site will be observed. 
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Individual students and instructional approaches will be observed. A 
student identification code will be assigied to each student. This will 
be his/her I.D. nunber for the entire study. A master list of students* 
names and IDs will be compiled and one copy given to the Chapter 1 teacher 
and another to the building principal. All data used fcy the DPI will have 
student IDs only. 

During an observation period/ the instructional approach will first be 
observed for a ten second time frame. Uien within the next five seconds^ 
this approach will be recorded. Next the first student will be dDserved 
for ten seconds. During that same ten second interval/ the instructional 
approach will also b3 observed as it relates to the student being 
observed. Both the student bdi^ior and the instructional approach will 
be recorded in the next five seconds. All students will be observed in 
this manner. 

When one complete cycle of the instructional approach and all students 
has been finished/ an amount of "wait time" will occur before another 
cycle is initiated. This "wait time" is built into the schedule so that 
each student and instructional approach will be observed fifteen tijnes 
during a class period. The "wait time" calculation is based on amount of 
time and nunber of students in the group. 

Sunmary of Observation Procedure 

I. Use Wait Time Matrix to determine number of seconds between 
observations. 

II, Observe the instructional approach for 10 seconds. 

A. Determine v^ther the instructional approach is interactive or 
non- interactive. 

B. Determine which of the categories is appropriate. 

C Record the appropriate code in the following five seconds. 

.IJI. CSDserve student #1 for ten seconds 

A. Determine if he/she is on-task or off-task. 

B. Determine which of the beh^ior categories is aj^opriate. 

IV. Observe the instructional approach. 

A. Determine whether the instructional approach is interactive or 
non-interactive in relationship to student #1. 

B. Determine which of the categories is ajpropriate. 

V. In the following five seconds^ record the appropriate student 
beh^ior code and instructional approach code. 

VI. Observe student #2 (Follow procedure outlined for student #1.) 

VII. Cteerve the rest of the students in the group in the same manner. 




Each site will be observed for one week at three different times 
during the school year. Hcwever, in each observation cycle, there are 
three weeks. Therefore, an observer will have three different 
"assignments" during a cycle. These same assignments will be repeated 
during the second and third cycle, ohe times are as follows: 

cycle I - October 31-Novanber 16, 1983 
cycle II - January 9-January 27 , 1984 
cycle III - March 12-March 30, 1984 

During the week of January 2, 1984 and during the week of March 5, 
1984, a one day follcw-up training session will be conducted for observers. 

GENERAL SCHOOL BEHAVIOR* 

1. Be considerate of school parking facilities. Be careful not to park in 
scmeone's assigned space, or to block other cars or entrances. If the 
school lot is small, or if there is any question as to whether or not you 
can park there, park on the street. 

2. When arriving, report to the office in the school buiding. Qqplain who you 
are, the purpose of your visit and hew long you will be there. 

3. Be as efficient as possible. Try to cause as little disruption to the 
school or classroan processes as possible. 

4. Maintain a professional approach at all times. Be friendly but not 
"chumny." Respect the teacher's position as the authority in his/her 
classroom. Respect the principal's authority as head of the school. And 
most of all, respect the enormous load that all schod personnel must 
handle. Try to be flexible and calm no matter what ha^sns. 

5. Maintain distance with the children. 

6. Be open and accepting of any suggestions made hy school personnel. If the 
suggestions do not conflict with the interest in obtaining precise cbta, 
try to be as acoontmodating as possible. 

7. If teachers or other school personnel ask questions about what you are 
doing, be polite, an»/er briefly, but do not discuss the matuer in detail. 

8. Never talk about a child, teacher, class or an/ other schod personnel 
while in the school. Remember that as an outsider, personal views will not 
be appreciated. 

9. Never discuss another school while on site. This can on]y lead to 
discomfort of those at the present site, as they will be concerned about 
what might be said about than at the iiext site. 

10. Be appreciative. Thank all involved for their time and assistance. 



* Leinhardt, Gaea, & Seewald, Andrea Mar, Student-Level Observation of 

Beginning Reading Manual . Pittsburg, Pennsylvania: Learning Research and 
Developnent Center, University of Pittsburg, 1980. 



r 



41 



50 



STUDENT CATEGORIES 



ON TASK 



Silent Reading 
(S) 



Comprehension 

(0 



I'ral Reading 
(0) 



Phonics 

(p) 




Study Skills 
(K) 



Language Arts 
_ (L) 



Test Taking 

(T) 



OFF TASK 




Out of Room 
(R) 



Being Disciplined 
. (D) 




Management 
(M) 



Academic Other 
(X) 



Academically 
Unoccupied 

(u) 



other Off Task 

(F) 



^2 
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INSTRUCTIONAL APPROACH 



CATEGORIES 



rivE 

SK 



eing 
CU) 



Presentation 
(PR) 



Questioning 
(QU) 



Monitoring 
(MO) 



Positive 
Reinforcement- 
Corrective 
Feedback 
(PF) 



Negative 
Reinforcement- 
CorrecLxve 
Feedback 
(NF) 



TIVE 
SK 



Discipline 
(PI) 



Management 
(MA) 



Other 
(CT) 



ERACTIVE 
I) 
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DEFINITIONS 



Student On-Task Behaviors 



Silent Ki\icUng: 



Oral__Rcadijng: 



Phonics: 



VocobularYt 



A student silently reading meaningful print in context. No 
voice sound is audible. Reading meaningful print in context 
implies that both the aoquisition of the meanings intended 
t^' the writer and the reader's own contributions in the form 
of interpretation^ evaluationr and reflection about these 
meanings are taKing plaoe^ Sane behaviors that may indicate 
a student is reading silently include: head mcvaientr eye 
mcvOTentf pointing to the printed words, lips forming words 
(but no sound emitted), and the relative position of the 
print and the student's head and eyes. 

Direct silent reading must deal with activities in which the 
material the studen^-s are using is more than one sentence in 
length. Reading pui^agraphs, passages, pages, etc., would 
constitute direct silent reading. 

A direct oral reading activity would have the same 
characteristics as direct silent reading, ' ut would be 
audible. A stadent reading aloud .ther chorally or alone 
would be a dire:c oral reading activity. 

Phonics is concerned with the speech sounds that correspond 
to letters, letter groups, and syllables in words; 
letter-sound correlation. Any activities in which 
letter-sound relationships ere the objective will be in the 
indirect phonics category. Sane gpneral phonics areas 
include: (1) consonant sound activit: \ (beginning, medial, 
ending), (2) consonant blend activities, (3) vowel 
acti/ities, (4) structural analysis ac*\vities (such as 
adding, s, to words) , (5) working with prefixes and 
suffixes, and (6) syllabication. 

Vocabulary activities will involve individual wordL. There 
are two types of vocabulary activities that will be 
observed. Both will be coded as 'Vocabulary, " with no 
designation of which type necessary. One type of vocabulary 
activity is learning to recognize words on sight (sight 
vocabulary activities) . These activities will involve 
working with ininediate recognition of words when they are 
seen. The other type of vocabulary activity involves 
increasing knowledge by learning the meaning of words. 
These activities will deal with individual words and their 
meaning* 

Examples of vocabulary activities would include doing flash 
card word drills, putting a list of words on the board and 
discussing cheir definition^, using desigiated words in 
sentences and playing "concentration" with word cards. 
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Canprehension: 



Study Skills; 



Language Arts ; 



Test Taking ; 



Conprehension can be defined as understanding and 
interpreting the meanings embodied in printed symbols. It 
is understanding v*iat an author has written. Following are 
examples of cmprdiension activities; 
" ani^ering questions relating to materials read 

- finding details 

- following directions 
" finding the main idea 

- sequencing 

~ anticipating meanings 

- drawing inferences 

- drawing generalizations 
~ evaluating 

" categorizing 
" predicting 

" distinguishing between fact and fancy 

Study skills can be defined as the functional skills of 
reading. These skills help students remonber what they have 
read. Following are examples of study skills: 
" ability to locate and interpret maps, graptis and 

charts 
" alfhabetizing 
" using the dictionary 
~ using the encyclopedia 
~ using other types of reference materials 

- using the card catalog 

- outlining 

" note- taking 

~ using book parts; cover, title page, title, author, 

publisher, etc 
" using table of contents 

" studying chapter heading, footnotes, section titles, 
etc 

~ using glossary, appendix, and bibliography 

Other language arts activities will be in this category. 
This includes listening, writing and speaking. When 
students are being read to by soneone or something (tape 
recording or reading machine) this will be considered 
"listening" and as sudi coded as language arts. Students 
who are e:q)laining scmething, retelling an incident or 
describing an event are classified as "speaking" and as such 
classified under language arts. This is different from 
compr^nsion activities wherein students are responding to 
questions concerning a particular pabsage that has been 
read. When students are writing a passage as part of a 
Chapter 1 assignment, this will also be considered language 
arts. This would be different than writing answers to 
comprdiension questions. 

This category includes the adnini strati on and taking of any 
type of test. It can be standardized, informal or criterion 
referenced. 
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student Off-Task Bdiayiors 

Absent s The student is not in school on this particular day. The 

observer will check after class with the teacher to find out if 
the student is truly absent fran schodL or just not in Chapter 1 
that 

Out of the Roan : 

The student is ir school , but not attending Chapter 1. The 
observe will ciieck after class with the teacher to find out if 
the student is siicply out of the room or absent tran school. 
This category will also be used if the student has left the room^ 
for example to use the restroan or get sane materials, during 
his/her observatic^ period. 

Being Disciplined ; 

Wnen the behavior of a student is not directly related to 
academi.c performance, and the teacher is attempting to alter that 
behanrior. 

When a student is waiting for or seeking assistance f ran the 
teacher. If a student has gone as far as he/she can on a lesson 
and needs further clarification, for example, he/she would be 
coded as Waiting." A beha^^ior can be coded as waiting only if 
it is caused ty the teacher. 

When a student is organizing or gathering materials for an 
assigiment or activity. Such things as finding the page to work 
on, getting materials f raa storage, and sharpening a pencil would 
be management activities. Also included would be listening to 
directions f ran the teadier. 

Academic Other ; 

When students are working on materials or assigments fran other 
classes such as mathematics, social studies or science. 

AcadCTiically Unoccupied ; 

Ary student beha^rior that is unrelated to academics. Included in 
this category would be socializing which is students interacting 
with other students or the teacher on a nonrtask topic. Also a 
student creating a distraction for other student (s) would be 
categorized as academically unoccupied. Examples of such 
disruptive heavier would include tapping a pencil on the table, 
hit^JLng another student, climbing under the table, and making 
faces. 

In addition to these examples, any time a student is inactive, 
observing activities in the roan or daydreaming, wo^ld be 
included in this category.' Staring into space, observing a 
disruptive student who is tapping his pencil on the table, and 
looking out of the window would be additional examples of a 
student being academically unoccupied. 



Waiting ; 



Management ; 
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other Off-T&sk t 

Ani^ activity that does not appear to fit in an^' other category 1 
The observer should maintain a listing of these behaviors. 

Interactive On-Task Behaviors 

Cueing ; When a teaclier indicates to student (s) v*iat is to be learned fcy 

giving directions related to the content of the lesson. In other 
wordSf explains viiat the student is expected to do in the 
learning process. Cueing goes beyond the management level ("do 
paoes 14 and 15 in your workbook") of handling naterials. Cueing 
will indicate what is to be leamedr \*at the student is to dOr 
an^or hew he/she is to do it. An example of cueing would be a 
teacher specifying objectives or tasks to be learned or 
practiced. He/she mi^t s^, "Ohe assignnents will be pages 14 
am 15 in the workbook and wilJ deal with beginning consonant 
sounds. " He/she then describes other activities the class has 
done relating to beginning sounds anj then relates this 
assignpent to their knowledge of the content. 

Another example of ageing would be demonstrating hcv to perform a 
task. At. illustration of this would be showing students how to 
pl^ a board game. 

Presentation ; When the teacher presents or e3?>lains a concept. The teacher 

presenting a lesson on the comparison of long vowel sounds would 
be an example of instruction. Making clarifications and engaging 
the students in drill or practice would also be categorized as 
presentation. Any other interactive instruction not included in 
the other categories will be included in this category as well. 

Positive Reinforcement Corrective Ffeedback ; 

Ary words or actions ty the teacher that will encourage the 
continuance of an activity ty the student will be included in 
this category. Praiser singling out a student for achievementr 
and smiles are seme examples, in additionr a teacher sharing 
specific information with a student about his/her academic 
performance in such a manner that it makes a positive impression 
on the student would also be included in this category. A 
statement such as "You've done very well today. On thin lesson 
you ccmpieted 9 of the 10 items correctly. This is much better 
than yesterday." would be another example. 
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Negative Reinf oroement- Ary words or actions by the teacher that will 
Corrective Feedback : discourage the continuance of an activity by the 

student will be included in this category. Statements 
such as "Why on earth did you do that?" and facial 
e3q)ressions such as a frown would be included. In 
addition, a teacher sharing specific information with a 
student about his/her academic performance in such a 
manner that it makes a negative impression on the 
student would be included as well. Sa/ing "You did 
poorly on this assigrment. You only got 2 of the 10 
items ri^t." would be another example. 



Questioning : When a teacher asks questions of whatever type to 

determine a student's knowledge. Any kind of 
canp!:ehension questioning would fit here. Management 
questions would not fit in this category. An example 
of a management question would be "Do you have your 
workbooks open?" 

Monitoring : When a teacher is not verbally interacting with 

studentSf and yet the teacher's attention is on the 
students. It could be that he/she is observing the 
students as they work independiently. 



Interactive Off-Task Behaviors 



Discipline : 



When a teacher is attempting to alter a student's 
behavior when that behavior is not specifically related 
to academic performance. 



Management : 



Other Off-Task: 



When a teacher gives directions without relating the 
task to the lesson content. Examples would include 
statements ty the teacher such as "get your books," 
"you will need pencils and paper" or "follow the 
directions in your packet. " 

Any teacher behaviors that do not relate to reading and 
are not specifically discipline or managerial r 
Examples would include establishing rapport or 
decorating the room for a holiday. 



NcMi- Interactive Behaviors 



Hiis category will be used whenever 
the teacher is not interacting with students. 
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OOEES 



Student 



Silent Reading 


= S 


Oral Reading 


= 0 


Hionics 


= P 


Vocabulary 


= V 


Canprehension 


= c 


Study Skills 


= K 


Language Arts 


= L 


Test Taking 


= T 


Absent 


= A 


Out of Roan 


= R 


Being Disciplined 


= D 


Waiting 


= W 


Management 


= M 


Academic Other 


= X 


Acadanically 
Unoccupied 


= u 


Other Off-Task 


= F 



Instruc±ional ^^roach 



Cueing 


= CU 


Presentation 


= m 


Questioning 


= QU 


Moiiitoring 


= MD 


Positive Feedback/ 
Reinforcement 


= PF 


Negative Feedback/ 
Reinforcement 


= NF 


Discipline 


= DI 


Management 


= MA 


Other 


= OT 


Non- Interact ive 


= NI 
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REMINEERS 



0/cLe Dates: 

cycle I: October 31 - Ncverober 18 

C/cle II: January 9 - January 27 

cycle III: March 12 - March 30 



Re- training Sessiai Dates: 

1 day during week of January 2 
1 day during week of March 5 



Coleen's Phone Nunbers: 

Work: (515) 281-3965 
Hone: (515) 233-1074 



Cdeen Not AvaHahle (1st cyde): 

Wednesday, Noveniber 2 (out of town, ccin leave message) 
ThursdcT^f Ncvanber 3 (out of town, can leave message) 
Friday, Ncvember 11 (office closed) 



Send Materials to This Address: 

Coleen MoQanahan 
Chapter 1 Reading Study 
Department of Public Instructioi' 
Grimes State Office Building 
Des Moines, Iowa 50319 
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Ajpendix C 



EbFINmONS OF GENERAL CHAEEER 1 
AND RBSULAR CLASSROCM VARIABLES 
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G ENERAL CHAPTER 1 VaRIABLES 

Group Size The nunber of stucSents receiving Chapter 1 instruction at 

the same time. 

Total Caseload A Chapter 1 teacher's total caseload. The "typical" mmber 

of students he/she maintains as the nonn. If 40 students is 
the goalr and 5 would mcve away during the year, and 5 
others were added, the caseload would still be 40 , rather 
than 50. 

Years of Teaching Es^rienoe 

The nunber of years an instructor had taught Chapter 1 (not 
total years of teaching) . The year of data collection was 
not counted. 

Teaching Certification 

The hi^est degree held as well as whether or not the Iowa 
reading approval had been obtained. 

Allocated Chapter 1 Class Time 



The amount of scheduled time for Chapter 1 classes. 

Actual Chapter 1 Class Time 

The amount of time Chapter 1 students actually spent in 
Chapter 1 class. 

Total Days of Instruction Possible 

The nunber of days between pretesting and posttesting that 
were available for students to attend Chapter 1 classes. 

Actual Days of Instruction 

The nunber cf days between pretesting and posttesting that 
students actually attended Chapter 1 classes. 

Days Absent The nunber of days students were absent between pretesting 

and posttesting. 

Days in School, but not Attending Chapter 1 Classes 

The nunber of days students were in school and scheduled for 
Chapter 1 classes, but did not attend. 
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Chapter 1 Participation Eligibility 



The scx>re fran a standardized test that maoa students eligible 
for the program, or the criterion reference method used to place 
students in the program. 

Pr etest Soore, Posttest Score and Gain 

Individual scores of each student in the project. 



RB3ULRR CLASSRDCM V/XIPBL ES 

Allocated Total Reading Time 

The total anount of time for which all reading classes were 
scheduled in the regular classrocitu 

Allocated Group Reading Time 

The amount of time for which Chapter 1 students' groups were 
scheduled for instruction in the regulrr cll cis sr oom. 

Qass Size The number of students in the regular classroom fran which the 
Chapter 1 students came. 

Groi^ Size The nunber cf studentr> in the Chapter 1 participants' reading 
group during regular classroon instruction. 

Teacher Certification 

The hi^est degree held as well as whether or not the Iowa 
Reading AK^rcval had been obtained. 
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Appendix D 



STUDENT SELECriCN »ID STUDENT ACHIEVEMENT INPORMftTICN 



ERIC 



DISQJSSirai OF STOEENT SELECTIDN 



m) STUIENT ACHIEVEHEaiT INEOMBVIKW 



All of the participating sites foil owed the oons{:raints associated with the 
Chapter 1 evaluation models A-i^ and only students who had valid see res were 
included in the analysis. Cat^aying with the owistraints of Model A-1, the 
following was the definition of a valid scores 

1. A standardized test was used. 

2. The student had both a pretest score and posttest score. 

3. The pretest and posttest were administered within 30 days of the 
nonuing date of the testo 

4 . The pretest score was not used for selecting students for the program. 

5. Tbe same test battery and subtest were used for pretesting and 
posttesting. 

6. The students were pretested in the fall and posttested in the spring. 

Because the majority of Qiapter 1 reading programs in the state of Iowa are 
pull-out programs, (vdiere the instructional approach is diacpostic-prescriptive 
and the instruction cocurred in a location other than the regular classroom) 
only students from tliese programs were included in this study. 

An established SEA procedure encourages local districts to select a test 
battery and subtest that are reflective of a match between tiiat test content 
and the content of the program being evaluated. Since Iowa has a computerized 
program available to convert raw scores to NCEs, local districts are required 
only to submit raw scores, and feedback reports are generated for than at the 
grade, building and district levels. Only the roost commonly used tests in the 
state are included in this system, due to the cost of obtaining and adding 
norms tables for each test batteiy involved. Districts generally use these 
included tests because feedback reports can then be obtained. These are the 
only constraints imposed by the SEA regarding instrunents used for evaluation 
purposes. 

Figures A-1 and A-2 shew the distribution of test batteries used by the 
sites in this study. A concern mi^t arise regarding the validity of the 
findings when noting the fact that nine different test batteries were used to 
gather data. However, in all but four cases, either the comprehension subtest 
or the total score was reported. Total reading scores are a composite of all 
areas on a test and largely over-lap with ccmprdiension scores. Hie four 
earaeptions used a vocabulary subtest score. 
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FIGURE A-1 

TEST BMTERIES USED BY SECOND GRAMI SITES 
FOR PRETESTINC5 AND POSTIESTING 



Test and Edition Year Peroent 



Gates MaoGinitie (78) 41 

Iowa Ttests of Basic Skills (78) 25 
Metropolitan Achievement Instructional (78) 10 

Stanford Diagiostic Reading Test (76) 10 

SRA Achievenent Test (77) 4 

California Achievement Test (77) 4 

Metroplitan Achievement Survey (78) 4 

Stanford Achievement Tfest (73) 2 



FIGURE A-2 

TEST BATTERIES USED BY FCXJRIH GRAEE SITES 
FOR IRETESTirc MD rOSTTEgTDG 

Test and Edition Year Peroent 



Gates MaoGinitie 48 

Iowa Tests of Basic Skills (78) 17 

Metropolitan Achievement Instructional (78) 13 

Stanford Diagiostic Reading Test (74) 7 

California Achievement Test (77) 7 

Metropolitan Acihievement Survey Battery (78) 4 

Metropolitan Achievement Test (70) 2 

Stanford Achievement Test (73) 2 
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^^ppendix E 



CHAPTER 1 TEACHER CRIENr;^TICN 
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CHAPTER 1 TEftCHER ORIENTATIDN 



It was natural that anxieties coid oc»ioerns would arise and should be 
addressed for personnel at the selected sites. T6 respond to these concerns, 
meetings for LEA st2rff who were involved in the project were planned and 
conducted by the Assistant Project Director and were held in three locations 
across the state prior to the initiation of the on-site observations. 
Approximately one hmdred peqple attended these meetings and 55 of the 64 
districts iivdved were represented. In addition, calls were received from 
four districts who were unable to attend because of unavoidable conflicts. 
Thc^5e meetings were essential to establish a positive enviroritient for the 
project. To preserve the integrity of the study, teachers involved were not 
informed of the specific bdiaviors that had been identified as observation 
categories. 

Information presented at these meetings included the following: 

A. History of Chapter 1 evaluation in Iowa 

B. Purpose of study 

C. Explanation of Secretary's Initiative for Chapter 1 Program 
Imprcvement 

D. Ratiorvde for on-site data collection 

E. Rationale for grades selected 

F. Site selection procedure 

G. Observer selection and training process 

H. Title frame 

!• Explanation of forms maintcdned ty teachers 

It was important to stress that the intent of this endeavor was to collect 
data, and that in no way were schools, programs, or teachers being evaluated. 
It was also stressed that all data would be reported in the aggretate, and that 
anonymity would be preserved. 

Teachers were asked to conduct classes in the same mannRr that they would 
if the observations weren't occuring. Tteachers were encouraged to maintain as 
"norm^d" a classroan routine as possible. While it is tempting to conduct 
npecial types of activities when visitors are in attendance it was emphasized 
that this W21S not desired for this stud/. 

Teachers were also asked to produce a sketch of their town or neighborhood 
along with a jet of directions that would aid the observer in finding the 
Bchool, parking, and where to go once in the building.* They were also asked 
to identify the students to be observed through the use of name tags so the 
observer could then establish an ID nunber for each child. The concept of a 
"master list" was explained; the master list being the names of the students 
and their assigned stud/ ID. Only observers. Chapter 1 teachers and the 
Ixjilding adninistrators had access to these master lists. They were never part 
of any DPI data. 

The teachers were given copies of any schedules that irrvolved them and were 
ackiKl to check for accuracy the tiines and grades listed. These schedules also 
indicated which of the three weeks the observer would be at their site. 



•Ooe sample on the following page. 
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5100-E4865M0/83 



Iowa Department of Public Instruction 
Chapter 1, ECIA Reading Study 
LEA Map 



(District) 




(Building) 


(Teacher's Name) 


(Building Principal's Name) 


(Building Address) 


Directons for finding building: 
(sketch a map) 




Directions for finding Chapter 1 Room: 
(sketch and/or written directio.is) 






Where should observer go if in building, 
but not observing a Chapter 1 class? 
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Appendix F 



CBSER/ER SELECTION AND 



72 

60 



ERIC 



CBSEtg/ER SELECTION MiD TRAINING 



To impLenent the on-site data collection strategy c£ this studyr personnel, 
who were referred to as "observers", were hired to ocmplete this task. 
Inaanuch as three distinct geographical areas cf the state were identified as 
the locations of the observation sites, the observers were hired f ran these 
three geographical areas as a cost contairroent effort of the, project. 
Advertising in the area newspapers as well as Iowa's major newspaper was 
intended to draw observer candidates fron each of the geographic regions and 
thereby reduce travel e^^nses. 

Three hundred eleven people responded to the advertisements; nineteen of 
whan were selected for training. The first step in the selection process 
involved elijninating those with no college work in education^ The fdlowing 
criteria were then applied to those remaining: 

1. Elementary school training 

2. Elementary school teaching experience 

3. Teaching e3?)erienoe in the area cf reading 

4. Iowa certification 

5. Residence within geographic proximity to study sites 

This objective process narrowed the list to 80 potential candidates. 
Polla/ing more scrutiny cf the application letters plus phone contacts to 
determine continued interest, 36 individuals were invited to personally 
intervia/ for the positions. The interviews were scheduled in the three 
geographic areas of the state represented ty the selected sites, and were 
conducted by the project director and assistant project director. Six 
interview locations were used. 

An interview form* was developed to systematize the uniformity of the 
intervia/ process between the interviewers. Bie form requested information 
about eroplcyment constraints, educational background, e^iperience in the 
teaching of reading, and references. In addition, six reading related terms 
were listed and the applicant was asked to define these terms. 

To achieve consistency, all of the definitions were rated ty one o* the 
intervia/ers. The "score" of the candidate on the terms, plus an inter/iew 
rating assigned on the basis of information and impressicMis given during the 
intervia/ were oonbined for a total interview rating. The ratings were ranked 
within each of the geographic areas, and 19 peopie were selected to participate 
in the training sessions. 

These 19 people participated in a three-day training session conducted ty 
tsao project staff members. Based on their performance at the training, 12 of 
these participants were selected as observers, and seven were selected as a 
reserve pool in the event a selected observer could not conpiete the 
ctoervations. 



*See sample at the end of this Appendix. 
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"A typical training program begins with an explanation of the purposes and 
theory involved in the given study and then moves on to an explanation of the 
categories and the rules for their use" vSelltiz et, al., 1959)* Observers 
were provided with a brief review of Chapter 1 in general, cuid a history of the 
Icwa evaluation effort in particular. Performance e:5)ectations of the 
observers were explained, which involved an explanation of the premises of the 
study and the definition of categories. 

Each of the student and teacher categories was presented and discussed in 
detail. Following this discussion, the trainers presented 91 oral examples of 
probable situations that would occur in ttie Chapter 1 classroons. The 
participants independently coded ti^ese situations; each situation was followed 
by a group discussion. 

The next step in the training sequence involved the use of 50 slides as a 
median to simulate Chapter 1 dassroan situations. In this segment of the 
training, the trainers showed the slides and described the specific situation. 
Again the participants coded the situations independently; each situation was 
followed by a groip discussion. 

The second day of training began with an e5?>lanation of the observation 
form (Appendix G). A silent video tape segment of an actual Chapter 1 
classroom was shewn during which trainers ej^lained what was happening cind how 
the observation form should be marked. 

The second video tape segnent involved .hree students. Enough of the tape 
was played to allow one cycle of observation to occur (all three students and 
the teacher) . The coding was done for the entire group by one trainer while 
the participants observed. After the cycle, the tape was stopped — the 
situation and coding revia/ed. Then another cycle of tliree students and the 
teacher was run and the same procedure follcwed. This v\'as done through the 
entire segnent. The entire segnent was then replayed fcr the participants to 
code. 

The next video tape segnent to be used was played for one cycle with a 
trainer maricing at the board and the participants markijig their observation 
forms independently. Pollowinq the cycle, discusston again occurred. Three 
more cycles were done in the same manner. Kie remainder of the segnent was 
played with the participants each independently codino their own form. Since 
timing would be a vital factor in the observation procedure, the ten-second 
intervals and five-second coding times were culled out by the trainer who was 
working the equipment. 

At this point a reliability check was run. The trainers had studied a tape 
segnent cycle by cycle, and agreed on the appropriate coding. The participants 
coded the same segnent and comparisons were made to determine 
trainer^participant reliability, as well as inter^rater reliability. 
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Another video tape segnent was played, three cycdes of which were coded at 
the board ty a trainer. Ihe remainder of the segment was played and the 
participants were to continue to code independently. Time intervals were 
called by a trainer. A review <£ the categories was conducted at thir^ pDin*:. 
This discussion helped to further clarify the coding process. A f a.nal tape 
segment was played that was totally handled ky the participants; both the 
coding and timing were their responsibility to achieve, itie last day of 
training consisted of additional tape segments and discussions. A final 
reliability check was run and procedural issues such as assignments, schedules, 
recordkeeping etc., were discussed. 

The principal analyst spent same time with the group addressing technical 
questions concerning observation procedures. In addition, he presented a 
matrix for wait time between observation cycles. A uniform nuT±)er of 
observations per class period, 15, had been established for all students who 
were to be observed. Since the number of minutes per class and the nunber of 
students in the class varied fron one site to another, the time matrix was 
developed for these varying situations. The matrix assured that observations 
were conducted ever an entire class period.* 

Each of the selected Chapter 1 reading groups were sch( *e obf^rved 

for an entire week at three different points in tijne to maxj le likelihood 
that the c*servations would adequately represent the instrur^ior^'l activities 
that occur between pretesting and posttesting. Observat :l1 ' cv . es" were 
identified as times near the beginning, middle and end ot he Cha er 1 
instructional year. The following observation schedule Wc. fc^.i :. d: 

cycle I - October 31-Navember 18, 1983 
cycle II - January 9-January 27, 1984 
Cycle III - March 12-March 30, 1984 

Each data collection cycle was limited to three weeks. On subsequent 
observation cycles, sites maintained th(^ir order of observatic^n estcblished in 
the prior cycle. This assured that the length of :'me between each obser/ation 
cycle was constant for each site selected. 

Each group was observed fifteen times (fives times each cycle). Each 
studer^ n each group was observed fifteen times during each of the 
obser\ tions, or 225 times. Make-up days were allowed during the fourth week 
it was noc possible to conduct all five of the enervations during the 
scheduled waek of a c\de.** 



*See inacrlx at the id of tUs Appendix. 

**An jxample ojl en observer' ; schedule can be found at the end of this 
Appendi c. 




Due to the short amount of time between notification z grant funding and 
the week in which it was necessary to begin the observaciuns, uie training 
session ended with understanding that each of the 19 pa*^ Acipants would receive 
a phone call the following day indicating whether or no- Lhey were to be an 
observer or part of the back-up pool. 

A review of observational procedures was conducte i t; rough a one-day 
workshop for observers and reserves prior to the second C4nd third observation 
cycles. The first follow-up training session focused prirtarily on a i .view of 
the categories. Again video tape segofjents of actui^ Ch .pter 1 classes were 
piayedf coded arid discussed, bach observer brought ii j^amples or situations 
they considered difficult to code, and these were c .^^ussed ty tlri groip. 
Another reliability check was run. The final follow-up t.rainin:: session used 
the same format and also included directions for sularittii^q f:j*:'. reports. In 
addition, the forms to be used for cdlection of reg:^^3r v u*^sr«_ am and 
additional Chapter 1 data were distributed and es^laintc!. 
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5100-E48619-i0/83 

Iowa Department of Public Instruction 
Chapter 1, ECIA Reading Study 
Application/Interview Form 

PRINT NAME AND ADDRESS 

Name 

AddressJ [ 

(Zip Code) 

Home telephone number: 

Other telephone number? :_ 

Do you have an automobile that you would be willing to use if employed? 

Are you willing to be away from home for periods of a week if 
necessary ? 

List Degrees: 

List Majors: 

List Reading Teaching Experience: 

1- 

2. 

3. 

4. 

Please Print Names and Address of References: 

1. 

2. 

3. 

4. 

Will you accept one-half day employment? 
Would you prefer one-half day employment? 
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Give a brief definition/explanation of the following teius: 
1. structural analysis - 



ERIC 



2. language experience - 



3. choral reading 



4. Spache 



5. basal - 



6 . C.A.I. 



What are your views (briefly) on the value of computer utilization for the 
teaching of reading in the elementary school grades? 



Are you aware of any personal future plans that would negate your being 
available for this effort through April 15, 1984? (Course work, other 

employment, moving, etc.) 

yes no 



Signed 
Date: 



Iowa Department of Public Instruction 
Chapter 1, ECIA Reading Study 
Wait Time Matrix 



Number of Students 



Minutes 
of Classroom 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


15 


32 


16 


0 
















20 


53 


37 


21 


5 














25 


75 


58 


H2 


26 


10 












30 


96 


80 


en 


H8 


30 


16 


0 








35 


117 


101 


85 


69 


53 


37 


21 


5 






HO 


139 


123 


107 


91 


75 


58 




26 


10 




H5 


160 


IHH 


128 


112 


96 


80 


en 


i<8 


32 


16 


50 


182 


166 


150 


133 


117 


101 


85 


69 


53 


37 
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OBSERVER 8 



Til 



LEA 



Iowa Department of Public Instruction 
Chapter 1, ECIA Reading Study 
Sample Observation Schedule 

BUILDING m i HQNDAY TUESDAY WEDNESDAY THURSDAY FRIDAY 



8*35- Q'On 

UtJJ 7 t\j[) 






0 

z 


X 


X 


X 


X 


X 




















9:55-10:15 


Site 5 


^18 


2 

L 


Y 
A 


Y 
A 


X 


X 


— — 

X 




















11:30-12:00 


Site C 






X 


X 


X 


X 


X 




















12:/(0- 1:00 


SiteB 


^18 


2 


X 


X 


X 


X 


X 




















2:25 - 2:55 


SiteD 

i — ' 


751 ^ , 


2 


X 


X 


X 


X 


X 
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Appendix G 



DftTA CXLLECriCN FORMS 

Chapter 1 Reading Stud/ Observation Record 
Pretest-Bosttest Information 
Student Attendance Record 
CLassroon/General Chapter 1 Information 
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m m 



I .'II 



i 



ft 
I" 

I 



mn\:\\ \ mmw siiiiiy obskrvation m:m 



(tiulMliiK) (lliillJliiK 



m □ □ 



1 1 1 1 

(tililtlrti (DlRliUI ID 
IHMHIIillHNAI 



iUnW (hoiiil (lomi-utru) ((iMhriirJs) (pwm) (laiit^iiflRf (machines) (reference (trails (worbheeu/ (Olhcr) 
•'tlvhln) exjiorlonce) books) books) workbooks) 



(]m\wr) (lVnrti.II))""""(Al(hT (A.II)^ (Cr) (Observation Date) (Observer ID) 

munth (lay year 



Mlint KM) (\mp. (C) 

Dill R. (0) Simly Hk. (K) 

riumlij (ri tJu^ Aril (I) 

Vofih. (V) Tfjl Irtk. |T) 




Sliulenl Oir-Tiak 

Absent (A) Hanage, (H) 

Out/RooBi (R) Acad, 0th. (X) 

Idni Dta. (D) Acad. Un. (I)) 

Wai ting (W) Other (F) 



Inst. Approach - Inter, On-Task 



Cueing (CU) Pos. Feedback (PF) 

Pres. (PR) Neg. Feedback (NF) 

Quest. (QU) 

Honlt. (HO) 



Diaclp. (DI) 
Hanage. (HA) 
Other (OT) 



(lU) 



ID 



Till 



TiM 



Till 



Inat. App. _ 

Inter. Off-Task hon-lr. Hive 
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Iowa Department of Public Instruction 
Chapter 1, iLClk Reading Study 
Pretest - Posttest Information 



® 



(5-a!) 

(9) 



1. 



3. 



□ 



(10-11) 4. 



(12-15) 



District Code 



Building Code 



Grade 



Teacher I.D. 



Pretest Information 

Name 

Form 

L eve 1^ 

Subtest 



(16) 



(17 -20) 



□ 



Score Type 



(NCE or Raw) 



7, 



Posttest Information 



Name 



Form 

Level 



Subtest 



Student ID RAW PRE RAW POST NCE PRE NCE POST MINUTES 



DAYS 



2UH 



42-62 
12-32 



33-53 



ma 



mdncn] 



□ 



54-7A 
12-32 



33-53 L 



54-74 



][ 



□ 



m [TB [ 



I. 



]m:] 



5100-E48698-10/83 



(District) 



(Building) 



(Grade) 



(Teacher) 



DPI USE ONLY 



(District ID) 



(Building ID) 



(Grade) 



(Teacher ID) 



(Student ij) 



Iowa Department of Public Instruction 
Chapter 1, ECIA Reading Study 
Student Attendance Record 



Sept. 



.Oct. 



Instructions: Please record the number of minutes each student attended 
Chapter 1 reading class each day. Begin recording 
on the day after the pretest was given. Finish the da, 
before posttest starts. 

If a student is not present, use the following codes to 
indicate the reason for an absence: 

A «= Absent from school 

0 " In school, but not in Chapter 1 

H = Holiday, conference day, any day, when school is not in 
session. 



Nov. 



^\^(^ IS !(> 21 28 29 30 3 4 5 6 7 10 II 12 13 \k 17 18 19 20 21 Ik 25 26 27 28 31 1 2 3 W 8 9 



«"v. Dec. jaiT 

~/v^vvvvvvv^^^^^^v^ ^^^^^^ vvi^^^^^^^^^Q^^^^^^^^^7^s^9 3o 2 3 45 ^ 9 



10U1213ieni8 19202324 25 25 213. 31 1 '2 3 O 8 , .Q inU5 15 1, . . „ ,o T"', , . , ■ 




9 . 10 11 14 15 16 17 18 21 22 23 24 25 

n 



iUT OFF DOTTED LI i BEFORE SUBMITTING TO DPI 
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5100-E55952-3/84 



Codes 

Due: May. IQail 
Pag e; 1 1 
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® 

(1-4) 

(5-8) 
(9) 

(10-11) 
(12-1i|) 

(15-17) 

(18-19) 
(20-21) 
(22-23) 
(24-25) 



Iowa Department of Public :^nstruction 
Chapter 1, ECIA Reading Study 
Classroom/General Chapter 1 Information 



District Code 
EZl Building Code 
I I Grade 
I I Teach^ir ID 



1 • Total allocated 
amount of 

classroom reading time 
(minutes per week) 

2. Amount of direct reading 
instruction time i ; the 
classroom for the^y^ I: :" ter 1 
students (minute?? per \x.rj^ ) 



3* Classroom class sl2C 
1 I I 4. Classv*oo '•oup sir:e 
I i I 5. Chapter i total oaseload 



J 6. Number of years instructor 
has taught Chapter I 
(excluding cm*rent year) 

CHECK ALL THAT APPLT: 





Chaoter 1 


Regular 
Classroom 






(26-27) 


□ 


□ 


7. 


Preprofessional certlflcatit 


(28-29) 


□ 


□ 


8. 


Bachelors degree 


(30-31) 


□ 


□ 


9. 


Masters degree (not in reading) 


(32-33) 


□ 


□ 


10. 


Masters degree Endorsement 38 
reading clinician 


(3U-35) 


□ 


□ 


11. 


Masters degree Endorsenent 54 
reading specialist 


(35-37) 


□ 


□ 


12. 


Approval 91 



S8 



student 
A 



38-51 



52-65 



66-79 



12-25 



26-39 



40-53 



54-67 L 



(S) .2-25 
26-39 



40-53 
54-67 C 



® 12-25 



26-39 



40-53 L 



54-67 C 
© 12-25 [ 



26-39 L 



40-53 C 



J 111 



] izr 
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] [ 



1=1 d 



] [ 



□ [ 



□ C 



J L 



Selection 
Score t 



] cm 



m MM c 



S9 



Norm 

Test IrState Test 
Sl£4& 2=Natlonal ^^S. 



] CI 



] nzi nni 



] cia 



□ I M I nm EH 



□zi 



□ 
□ 
□ 
□ 
□ 



□ 
□ 
□ 



□ 

□ 



□ 



TABLE A-1 



CXDRRELATICN OF GAIN AND PRETEST 
WITH STUDENT VARIABLES, 
INSTHJCTICmL APPROACHES AND STFUCTORAL VARIABLES 

INDIVIDUAL LEVEL ANALYSIS 



Grade 2 Grade 4 

Variables Gain Pretest Gain Pretest 



Student On-Task 



Oral Reading -24 

Silent Reading -02 

Phonics 19 

Vocabulary 16 

Comprehension -05 

Language -20 

Study Skills -03 

Test Taking -14 

Student Off-Task 

Being Disciplined 04 

Waiting 56 

Management 15 

Academic Other 14 

Academically Unoccupied -10 

Other Off-Task 39 

Teacher Interactive On-Task 

Cueing -01 

Presenting -01 

Questioning 08 

Monitoring 25 

Positive Feedback 25 

Negative Feedback 47 

Teacher Interactive Off -Task 

Disciplining -02 

Management 31 

Other Off-Task -19 

Teacher Non-Interactive 18 

Structural 

Pretest Score -59 

Selection Score -33 
Actual Amount of Instruction 14 

Group Size 03 

Ceiseload 31 



Chapter 1 Experience of Teacher 16 



10 -01 00 

-01 00 06 

-17 00 -13 

-11 20 02 

14 -01 18 

15 14 01 
-04 -03 11 

16 01 04 



-03 -02 -33 

-19 C7 03 

08 02 16 

-28 03 -01 

13 00 -11 

-17 -16 10 



14 -05 14 

14 -05 14 

-01 02 04 

-04 -01 09 

-11 09 06 

-22 -20 00 



-02 01 -29 

-08 00 13 

16 -02 01 

-i 01 04 



1.0 -43 1.0 

29 -06 15 

01 -03 11 

03 -04 08 

-J.O 10 -21 

07 15 04 
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TABLE A-2 



ODRREIiMPICN OF ACTUAL* AMOUNT OF ON-TAEK 

TI^E Wm SlUEENT ACTIVITIES, 

INSTFUCTIONAL APPROACHES i^TO STFUCIURAL CHARACTERISTICS 

GROUP LEVEL ANALYSIS 
GRADE 2 



Student On-Task 



Actual On-Task Time 



Oral Reading 
Silent Reading 
Phonics 
Vocabulary 
Canprehension 
Language 
Study Skills 
Test Taking 

Student Off-Task 

Being Disciplined 

Waiting 

^janagement 

Academic Other 

Academically Unoccupied 

Other Off-Task 

Teacher Interactive On-Task 

Cueing 
Presenting 
Questioning 
Monitoring 
Positive Feedback 
Negative Feedback 

Teacher Interactive Off-Task 

Disciplining 
Management 
Other Off -Task 

Teacher Not- Interactive 



.26 
.13 
.60 
.18 
.02 
.32 
-.12 
-.07 



.21 
.36 
.20 
.01 
.10 
.35 



.28 
.28 
.30 
.60 
.42 
.45 



.15 
.33 
.20 

.32 



Structural 



Pretest Score 

Actual Anount of Instruction 

Group Size 

caseload 

Chapter 1 Experience of Teacher 
Gain 



-.09 
.82 
.05 
.22 
.06 
.03 



♦Aggre^ted observations of percent of on-task time multiplied by the amount of 
instructual ea^josure between pretesting and posttesting. 
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TABLE A-3 



ODRRELATIOJ OF ACTUAL* AMOUNT OF ON-TASK 
TIME Wm SIUEENT ACTIVITIES, 
INSTHJCriONAL APHOACHES AND STKJCIURAL CHARACTERISTICS 

GROUP LEVEIj ANALYSIS 
GRADE 4 



Student On-Task Ac±ual On-Task Time 

Oral Reading .17 

Silent Reading .26 

Hionics .39 

Vocabulary .45 

Canprdiension .40 

Language .52 

Study Skills .28 

Tfest Taking .11 

Student Off-Tiask 

Being Disciplined -.09 

Waiting .26 

Management .49 

Academic Other -.11 

Academically Unoccupied -.09 

Other Off-Task .14 

Teacher Interactive On-Task 

Cueing .26 

Presenting .26 

Questioning .35 

Monitoring .50 

Positive Feedback .26 

Negative Feedback -.03 

Teacher Interactive Off-Task 

Disciplining -.08 

Management .46 

Other Off-Task -.03 

Teacher Non- Interactive -.17 

Structural 

Pretest Score .16 

Actual Amount of Instruction .83 

Group size .26 

Cciseload -.04 

Chapter 1 Ej^jeriance of Teacher .00 

Gain .10 



♦Aggregated observations of percent c£ on- task time multiplied ty tlie amount of 
instruct JBl eaqxjsure between pretesting and posttesting. 
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TABLE A-4 



OORRELATICN OF TEACHER OTHER OFF-TASK 

VARIABLE Wm STOEENT ACTIVITIES 
AND TEACHER INSTFDCTICNi^ APFrOACHES 

GROUP LEVEL ANALYSIS 
GRADE 2 





Teacher Other Off-Task Activities 


UroJ. KcaCuJig 


.23 


Silent Reading 


-.02 


Hionics 


.04 


Vocabulary 


.19 


Ccinpr€tension 


-.25 




• 32 


oXuuy bKlllS 


•04 


i<coC laKing 


-.06 


otuoenc utt-"uBSK 




Being Disciplined 


-.03 


Waiting 


-.05 


?1anagement 


.j7 


/vcaocniic Uuner 


.jl2 


Academically Unoccupied 


.66 


Other Off-Task 


.41 






Cueing 


.13 


Presenting 


.13 


Questioning 


-.24 


Monitoring 


.12 


Positive Feedback 


.22 


Negative Feedback 


.03 


Tfeacher Interactive Off-Task 




Disciplining 


.55 


Managanent 


.27 


Teacher Non- Interactive 


.55 



TABLE Ar5 



(DRRELATICN OF GROUP SIZE WITH 
STUEENT ACriVITIES, TEACHER INSIKUCriDNAL APFROACHES 
AND STKJCIURAL CHARACTERISTICS 

GROUP LEVEL ANMiYSIS 
GRi^DE 2 



Student On-Task 



Oral Reading 
Silent Reading 
Hionics 
Vocabulary 
Canprehension 
Language 
Study Skills 
Test Taking 

Student Off-Task . 

being Disciplined 

Waiting 

Management 

Acadeniic Other 

AcadHnically Unoccupied 

Other Off-T&sk 

Teacher Interactive On-Task 

Cueing 
Presenting 
Questioning 
Monitoring 
Positive Feedback 
Ifegative Feedback 

Teacher Interactive Off-T^s': 

Disciplining 
Management 
Other Off-Task 

Teacher Non- Interactive 

Structura l 

Pretest Score 

Actual Amojnt cf instruction 
Caseload 

Chapter 1 Experience of Tteacher 
Gain 



Group Size 



-.20 
.21 
.03 
.06 
.00 
.03 
.01 
.04 



.23 
.22 
.07 
.00 
.20 
.05 



.28 
.28 
.15 
-.04 
.20 
.10 



.27 
.14 
.08 

.14 



.04 
.00 

.23 
.19 

.03 
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TABLE Ar6 



HCW A TEN PERCENT INCMIASE IN TIME 
ALLOTTED ECR SKaFIC ACTIVrriES CMi INFLUENCE GAIN 

INDIVIDUAL LEVEL ANALYSIS 
GRAEE 4 



Actual 10% Stand. NCE 

Variable Time Increase Beta Est. Probability Change 



Vocabulary 415 42 .0005 .13 .0092 .21 

Student Other Off-Task 05 10 -.02 -.15 .0033 -.20 

Negative Feedback 30 3 -.06 -.13 . 0130 -.18 

Being Disciplined 3 .3 -.14 -.13 .0146 -.04 

Langiage 240 24 . 0006 .11 .0259 .14 

Positive Efeedback 30 3 .06 .09 .0917 .18 



TABLE A-7 



HCW A TEN PERCENT INCREASE IN THE 
ALLOnED FOR SEEaEIC ACTIVrriES CAN INELUENCE GAIN 

indivuxjMj level analysis 

GRADE 2 



Actual 10% Increase 





Time In 


In 




Stand. 




NCE 


Variable 


Minutes 


Minutes 


Beta 


Est. 


Prdaability 


Change 


Oral Reading 


115 


12 


-,04 


.22 


.0001 


-.48 


Teacher Other Off-Task 90 


9 


-.03 


-.26 


.0001 


-.27 


Teacher Management 


415 


42 


.008 


.13 


.0039 


.34 


Phonics 


720 


72 


-.005 


-.16 


.0005 


-.36 


Monitoring 


620 


62 


.003 


.13 


.0116 


.19 


Providing Positive 










.0093 




Feedback 


30 


3 


.07 


.11 


.21 


Being Disciplined 


5 


.5 


-.10 


-.06 


.1006 


-.05 


Vocabulary 


420 


42 


.003 


.06 


.1267 


.13 



.96 

8?' 
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